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I.  INTRODUCTION 

A.  An  Overview  of  Crew  Resource  Management  (CRM!  Integration 

The  integration,  and  assessment,  of  Crew  Resource  Management  (CRM)  and  flight  control 
skills  has  received  considerable  attention— and,  a  fair  share  of  concern  and  skepticism— over  the  past 
few  years.  As  one  response,  the  ATA  formed  an  air  carrier/FAA/ academe  working  group  to  deal 
with  this,  and  other  CRM  issues,  in  1990. 

The  main  issues  in  doing  a  simultaneous  and  integrated  assessment  of  CRM  and  flight  control 
performance  revolve  around: 

1.  Identifying,  developing  and  validating  the  observable/rateable  performance  behaviors 
that  define  CRM. 

2.  Developing  a  behaviorally-anchored  scale,  or  set  of  scales  by  which  to  assess  these 
CRM  performance  behaviors.  There  is  also  the  problem  of  developing  a  set  of  crew 
performance  behaviors  for  the  technical  flight  control  skills  similar  in  format  to  the 
CRM  performance  markers;  this  set  would  then  be  used  in  any  attempt  at  the 
integration  with  the  CRM  behaviors. 

3.  Developing  an  analytic  paradigm  which  could  both  identify  and  demonstrate  ( what 
were)  the  CRM  performance  behaviors  embedded  in,  and  intrinsic  to,  the  flight  control 
skills  necessary  for  safe,  efficient  missions.  Such  a  paradigm  must  be  able  to 
analytically  show  where  the  integration  of  CRM  and  flight  control  skills  occurred,  i.e. 
where  during  the  accomplishment  of  which  maneuvers/tasks/sub-tasks.  The  model 
should  be  capable  of  dealing,  on  a  specific  level,  not  only  with  different  aircraft  types, 
but  also  with  different  environmental  conditions  and  with  the  different  SOP’s  in  use 
with  the  different  air  carriers. 

4.  Finally,  any  model  or  paradigm  developed  needed  to  be  both  operationally-oriented  and 
very  accurate.  This  is  because  any  CRM  integration  paradigm  would  immediately 
confront  a  mind-set  that  has  evolved  in  the  development  and  "selling"  of  CRM  and 
from  the  idea  of  the  existence  of  "soft"  (as  opposed  to  "hard")  piloting  skills. 

Historically,  much  of  the  original  impetus  for  the  formalized  CRM  research  and  development 
came  from  a  series  of  commercial  aviation  accidents.  In  these  well-publicized,  fatal  mishaps, 
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neither  aircraft  malfunction  nor  maintenance  were  the  causal  factor(s).  Rather,  communication, 
command,  leadership,  and  other  psycho-social  factors  were  called  out  as  the  real  problems. 

There  were  several  people  who  had  recognized  these  issues  prior  to  the  string  of  accidents  and 
who  then  stepped  forward.  People  such  as  John  Lauber,  Clay  Foushee  and  Bob  Helmreich  began 
the  research  and  development  (or,  in  some  cases,  continued  their  existing  work)  which  brought 
forth  much  of  initial  CRM.  However,  almost  from  its  inception,  CRM  was  seen  as  somewhat  of  a  #• 

stand-alone  and/or  a  "fix"  for  a  certain  type  of  accident.  Since  many  of  the  people  who -came  into 
the  field  next  were  from  the  discipline  of  organizational  development,  there  also  grew  a  perception 
that  CRM  was  involved  only  with  soft,  squishy  "crow  like  a  rooster"  activities  and  skills.  Lastly, 
in  «m  effort  to  encourage  the  air  carriers  to  give  .CRM  training  and  to  encourage  line  pilots  to  take 
this  training  in  a  non-jeopardy  context,  the  FAA  began  to  grant  a  recurrency  waiver  if  CRM 
training  was  taken.  This  action  by  the  FAA  had  the  additional  effect  of  reinforcing  a  view  that 
CRM  training  was  a  separate,  stand-alone  activity. 

In  fact,  and  in  too  many  places,  CRM  indeed  was  a  stand-alone  piece  of  training  with 
insufficient,  or  no  effective  input  from  the  pilot  community  in  development  and  delivery-and,  with 
little,  or  no  effort  to  relate  CRM  to  traditional  flying  skills.  It  is  easy  to  see  how  a  mind-set  came 
into  place  which  incorporated  all  of  these  data. 

Still,  many  pilots  and  researchers  with  flying  experience  knew,  almost  intuitively,  that  CRM- 
type  skills  were  part  and  parcel  of  what  good  aviators  did,  and  had  always  done  and  taught.  As 
time  went  on  and  CRM  grew  and  evolved,  a  view  that  the  CRM  skills  had  mistakenly  and 
artificially  been  separated  from  the  flight  control  skills  also  began  to  grow.  By  the  early  1990s, 
with  on-going  research  and  development  in  CRM  and  the  FAA’s  Special  Federal  Aviation 
Regulation  (SFAR)  on  Advanced  Qualification  Programs  (AQP)  for  aircrew  providing  impetus, 

CRM  integration  efforts  began  in  earnest. 

Issues  I.A.l  and  I. A. 2  above  have  been  extensively  worked  by  Dr.  Robert  Helmreich,  in 
conjunction  with  several  major  air  carriers.  At  this  time,  he  has  developed  a  complete  set  of  ► 

flightcrew  CRM  performance  markers  (he  terms  them  "CRM  behavioral  markers")  with 
behaviorally-anchored  rating  scales.  In  a  NASA/FAA/University  of  Texas  project,  Helmreich  has 
worked  with  several  air  carriers  on  research  which  involves  the  use  of  these  markers  in  Line- 
Oriented  Flight  Training  (LOFT). 

In  1991,  Captain  Kevin  Smith  (United  AirLines)  and  Jan  Demuth  (FAA  Flight  Standards) 
developed  an  initial  set  of  performance  markers  for  the  Technical/Flight  Control  skills.  Both  the 
CRM  and  the  Technical  sets  of  markers  were  used  in  the  next  step  of  CRM  Integration:  the  attempt 
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at  developing  an  analytic  paradigm  (i.e.  issue  I.A.3).  Kevin  Smith  created  the  framework  for  a 
model  which  does  demonstrate  that  the  CRM  human  factors  skills  and  the  Technical/Flight  Control 
skills  are  interrelated,  interdependent,  and  often  simultaneous  in  execution-that,  for  safe  and 
efficient  flight,  CRM  is  integral  to  flight  control,  and  vice-versa.  This  model  is  called  the  Mission 
Performance  Model  (MPM). 

Captain  Smith,  a  member  of  the  ATA  committee  on  CRM,  then  worked  with  Bob  Helmreich, 
Jan  Demuth  and  Ron  Lofaro  to  extend,  articulate  and  apply  the  MPM  to  actual  flight  maneuvers, 
such  as  an  engine  out  at  VI,  with  a  turn  procedure  required  by  the  terrain.  Concomitant  with  this 
effort,  the  ATA  CRM  group  was  working  on  a  major  revision  to  the  FAA’s  1989  Advisory 
Circular  on  CRM  ( '#  120-51).  As  several  key  players  on  the  MPM  were  also  involved  with 
drafting  portions  of  the  CRM  Advisory,  there  was  intense  cross-fertilization,  to  the  benefit  of  both 
efforts.  The  revised  Advisory  Circular  is  now  in  the  process  of  final  review/changes  by  the 
industry  after  which  it  enters  the  FAA  approval  process.  The  possible  publication  date  is  mid 
1992. 

B.  Brief  History  of  the  Workshop 

In  June/July  of  1991,  Ron  Lofaro  and  Kevin  Smith  began  to  see  some  need  for  a  select  group 
of  experts  to  be  convened  and  charged  with  several  tasks— not  the  least  of  which  would  be  an  in- 
depth  exploration  of  the  MPM.  At  several  ATA  working  group  and  sub-committee  meetings 
(Washington,  Austin)  during  late  1991,  the  idea  of  a  "select  team"  workshop  on  CRM  Integration 
and  the  MPM  was  brought  forward.  Many  people  expressed  interest  and  indicated  their 
availability  as  workshop  participants.  Dr.  Lofaro  volunteered  his  services  as  a  designer/ facilitator 
and  the  facilities  of  MiTech,  Inc.,  in  Washington,  D.C. 

The  workshop  was  conceived  of  as  having  two  phases:  in  Phase  I,  potential  participants  would 
be  given  reading  materials  and  assignments  as  a  preparation  for  a  workshop;  Phase  II  was  to  be  the 
actual  2'A.  day  workshop.  The  Phase  I  reading  materials  would  contain  discussions  and 
explanations  on  CRM  Integration  and  the  MPM,  "broad-brush"  Phase  II  objectives,  a  tentative  list 
of  participants  for  Phase  II,  and  the  time-frame  (19-21  November). 

The  Phase  I  materials  were  prepared  as  a  booklet,  coordinated  and  sent  to  the  prospective 
Phase  II  participants.  Incorporated  into  this  Phase  I  booklet  were  the  latest  drafts  of  both  the  CRM 
Advisory  and  of  the  CRM  and  Technical  crew  performance  behavioral  markers.  The  Phase  II 
workshop,  and  all  materials  prepared  prior  to,  and  for  it,  are  the  content  matter  of  this  report. 
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II.  THE  WORKSHOP 

A.  Participant  Profile 

There  were  six  major  air  carrier  line  pilots,  five  of  whom  were  also  training  program 
managers/directors— usually  in  CRM  or  AQP. 

There  were  two  ISD  experts,  one  from  the  Naval  Post  Graduate  School’s  Aviation  Safety 
branch  and  one  from  the  FAA’S  Flight  Standards’  AQP  organization,  as  well  as  the  Chief  Pilot 
(Flight  Training)  of  The  Boeing  Company.  There  was  an  FAA  research  psychologist  who 
functioned  as  the  workshop’s  designer  and  facilitator.  Additionally,  Capt.  Kevin  Smith  of  UAL 
presented  and  stayed  for  all  of  Day  I,  and  Mr.  Jan  Demuth,  FAA  Flight  Standards’  CRM 
Coordinator,  was  there  for  Day  I  and  all  of  the  half-day  on  Day  III. 

The  participating  pilots  averaged  15  years  as  a  pilot  with  their  current  organization  (ranging 
from  4  to  25  years),  and  held  qualifications  in  an  average  of  3  aircraft  (see  Appendix  F). 

B.  Procedures/Process  and  Materials 

In  early  October,  1991,  each  participant  received  a  Phase  I  Booklet  (See  Appendix  A).  This 
package  gave  a  brief  background  of  CRM  integration,  an  overview  of  the  MPM,  the  latest  draft  of 
the  (revised)  FAA  Advisory  Circular  on  CRM  and  the  crew  performance  markers— CRM  and 
Technical— with  rating  scales.  This  was  a  56-page  booklet. 


NOTE:  For  this  report,  we  have  removed  the  draft  version  (Draft  2.5)  of  the  CRM  Advisory 
Circular  which  was  a  part  of  the  Phase  T  booklet.  Rather,  we  have  included  the  latest  draft 
(Version  3.4)  as  Appendix  C. 


The  participants  were  asked  to  read  and  to  familiarize  themselves  with  all  of  the  Phase  I 
materials  and  to  complete  Objectives  IA  and  IB  individually. 

Objective  I: 


A.  To  examine  the  behavioral  markers-both  CRM  and  Technical. 
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B.  To  examine  the  two  behaviorally  anchored  rating  scale(s)  for  the  above--  both  of  these 
can  be  (initially  and  individually)  a  "homework"  part  of  Phase  I;  the  finalization  via 
group  work  we  will  do  at  the  workshop. 

Objective  II:  To  complete  the  mission  performance  model  by: 

Al.  Developing  a  criticality  scale/ rating  methodology  (or,  other  method)  for  the 

identification  and  criticality-rating  of  the  sub-tasks/elements  on  any  mission  task. 

A2.  ALTERNATIVELY,  comparing/cr  iitrasting/modifying/integrating  (--whatever)  the 
MPM  and  Figure  2  in  the  latest  draft  of  the  CRM  Advisory.  (Remainder  of 
objective  is  somewhat  the  same.) 

B.  Exercise  the  model-use  3  tasks  per  phase  of  flight  (normal;  abnormal/  emergency; 
"other")  as  an  initial  review/critique  and  modification;  as  a  check  on  the 
completeness  of  the  task  decomposition  via  criticality-rating  (or,  whatever  other 
method  was  used). 

C.  Re-exercise  the  model  to  check  any  mods  that  were  made  (—same  format  as  in 
II. B,  IV.. 5  as  to  the  number  of  tasks,  etc.). 

D.  Identify/articulate  any  rules  and  algorithms  embedded  in  the  model. 

E.  Develop  scales/rules  for  evaluating  the  existence/strength  of  CRM  and  technical 
skill  interrelationships-task/sub-task  level. 

Objective  HI:  To  identify/examine  the  potential  uses  of  the  model-with  attention  to: 

A.  Aircrew  CRM  Training  development; 

B.  LOFT  scenario  development  and/or  evaluation; 

C.  Instructor  Pilot/Evaluator  training; 

D.  CRM/LOFT  evaluation. 

This  Phase  I  "homework  assignment"  was  in  preparation  for  the  Phase  II  workshop,  which  was 
also  referred  to  in  the  Phase  I  booklet. 
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During  the  next  four  weeks,  conversations  occurred  between  Ron  Lofaro  and  all  the 
participants.  These  telephone  talks  covered  a  variety  of  questions  and  concerns  on  the  Phase  II 
effort. 


Day  I: 

On  November  19,  1991,  the  Phase  II  workshop  was  convened  at  0900  at  the  Headquarters  of 
MiTech,  Inc.,  in  Washington,  DC.  This  company  has  a  support  contract  with  the  FAA/ARD-200 
and  their  headquarters  were  chosen  because  of  the  facilities  available,  ease  of  travel  (the/  are  one 
block  away  from  the  Metro  stop  at  Union  Station),  and  to  provide  a  neutral  atmosphere  for  the 
wo  kshop.  Ron  LoFaro  welcomed  the  participants,  spoke  on  the  goals  of  the  workshop,  then  on 
administrative  and  other  details,  and  distributed  the  Phase  II  booklet  (see  Appendix  B)  and  the 
latest  draft  of  the  FAA  CRM  Advisory  Circular  (see  Appendix  C). 

Jan  Demuth  offered  a  welcome  on  behalf  of  himself  and  his  (Flight  Standards)  organization. 
He  then  gave  some  views  on  the  MPM.  Included  in  these  were:  (1)  his  hopes  that  the  workshop 
was  not  construed  as  a  sub  rosa  effort  to  make  the  MPM  an  industry  standard,  use  of  the  MPM  in 
CRM  or  AQP  was  not  "required;"  (2)  the  MPM  should  be  seen  as  a  jumping-off  point  to  many 
different  training  and  assessment  endeavors;  (3)  the  MPM  could  logically  and  profitably  be  looked 
at  as  part  of  larger,  and  not  yet  developed  model(s).  Mr.  Demuth  then  offered  some  insights  on 
the  MPM,  as  it  now  exists,  in  relation  to  AQP.  He  did  want  an  awareness  that  the  MPM,  is  not, 
by  itself,  a  front-end  analysis  (FEA)  tool  as  it  does  not  "fold  in"  a  condition  set.  In  that  sense,  the 
MPM  (as  is)  should  not  be  used  to  write  critical  performance  objectives.  Finally,  Jan  offered  his 
insight  that  the  MPM,  as  it  is  developed,  would  only  be  able  to  deliver  its  full  value  if  it  were 
computerized/automated.  He  then,  again,  spoke  of  the  MPM  as  a  starting  point  and  one  which 
holds  great  promise. 

During  the  next  five  hours,  Captains  Kevin  Smith  and  Bill  Hamman  of  United  Air  Lines  gave 
lengthy  expository  presentation:  on  the  MPM  and  CRM  Integration.  Captain  Hamman  also 
showed  a  45-minute  videotape  of  a  portion  of  a  LOFT  scenario.  In  this  video,  the  LAX  Civit 
profile  descent  was  being  flown,  while  the  crew  was  distracted  by  a  cabin  problem.  The  video  was 
accompanied  by  an  analytic  decomposition  of  the  tasks  and  sub-tasks  required  to  successfully 
perform  this  profile.  This  analysis  had  been  performed  using  the  MPM.  This  tape  demonstrated 
that  the  model  did  capture  all  the  critical  tasks  required  to  successfully  perform  the  maneuvers  as 
well  as  providing  an  assessment  schema  which  could  differentiate  levels  of  crew  CRM  and 
Technical  performances.  The  video,  with  the  handout,  seemed  to  show  a  prima  facia  validity  for 
the  MPM. 
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There  were  consensual  modifications  made  to  Objectives  I  and  II.  In  Objective  1,  step  A,  it 
was  decided  to  look,  not  only  at  the  markers  themselves,  but  also  at  the  categories  and  clusters  to 
which  they  belong.  It  also  was  decided  to  discuss  and  critique  the  (latest)  draft  version  of  the  FAA 
Advisory  Circular  on  CRM  which  was  provided  by  Jan  Demuth  on  Day  I. 

In  Objective  II,  step  A1  (a  criticality/rating  scale),  it  was  decided  to  discuss  the  concept, 
rationale  and  use  for  the  use  of  this  type  of  scale  in  the  identification  of  the  sub-tasks  which  are 
critical  to  a  flight  task/ maneuver.  Objective  II,  step  A2,  was  scrubbed  altogether.  In  Objective  II, 
the  exercising  of  the  model  and  beyond  (steps  B  through  E)  was  not  done.  There  were  several 
reasons,  the  major  one  being  a  feeling  that  not  until  Day  1  had  the  MPM  begun  to  be  truly 
understood  by  the  workshop  participants.  The  MPM,  or  portions  of  it,  had  been  briefed  to  the 
majority  of  the  workshop  participants  at  ATA  meetings  in  May,  September  and  November  1991. 
However,  there  was  little  or  no  time  for  full  discussion  at  these  ATA  meetings  as  the  agendas  were 
full  and  required  that  Q  and  A  were  held  down  considerably. 

During  Day  I,  both  Kevin  Smith  and  Bill  Ham  man  presented  the  MPM  and  its  application  in 
great  detail,  with  full  opportunity  for  questions,  explanations,  discussion.  Even  for  those  who  had 
felt  familiar  with  the  model,  these  presentations  incorporated  new  work  done  on  the  model  (after 
the  last  ATA  sessions  and  prior  to  the  workshop)  which  lead  to  some  reflection  and  re-thinking  on 
the  MPM.  Hard  copies  of  these  briefings  are  at  Appendix  A  and,  along  with  the  Phase  I  materials 
on  the  MPM,  should  provide  insight  and  understanding  to  the  reader  as  they  did  to  the  participants. 

During  the  last  hour  of  Day  I,  Ron  Lofaro  led  a  detailed  discussion  on  the  workshop 
objectives  as  well  as  the  procedures  for  accomplishing  them.  An  0830  stan-ume  for  Days  II  and 
III  was  agreed  on  and  the  participants  were  requested  to  review  their  "homework"  assignment. 

Day  II: 

A  brief  period  of  time  was  spent  by  Ron  Lofaro  in  synopsizing  Day  I  and  in  making  team 
assignments.  There  were  two  "work-teams"  of  five  members  each;  the  entire  group  was 
considered  as  the  "intact"  team.  Objectives  I,  A&B,  were  separately  worked  on  by  the  working 
groups.  After  lunch,  Dr.  Lofaro  convened  the  intact  group,  both  work  groups  presented  their 
findings  and  a  discussion  ensued  which  led  to  a  consensus  as  to  these  objectives. 
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NOTE:  The  original  planning  of  the  workshop  was  to  use  Ron  Lofaro’s  "modified  small  group 
Delphi"  paradigm*  to  do  the  objectives.  However,  due  to  the  length  of  the  Day  I  presentations 
and  discussion,  time  constraints  did  not  allow  for  use  of  all  paradigm  components,  nor  for  the 
successive  iterations  needed  to  achieve  all  criteria  of  the  paradigm.  On  the  other  hand,  the 
workshop  was  able  to  use  many  of  the  paradigm’s  processes  and  Ron  Lofaro  was  available  to 
facilitate  the  entire  effort.  As  was  shown  in  the  participants  response  satisfaction  questionnaires, 
they  felt  very  comfortable  with  both  the  process  and  the  results.  (See  Appendix  F.) 

The  remainder  of  Day  II  saw  a  modification  to  the  schedule  and  objectives  in  that  the  group 
immediately  decided  they  wanted  to  do  a  final  critique  of  the  Draft  CRM  Advisory  Circular  and 
then,  quickly,  and  as  an  intact  group,  deal  with  Objective  III.  This  was  accomplished  and  the 
group  broke  at  1600. 


Day  HI: 

This  was  a  half-day  session  and  began  at  0800.  Jan  Demuth  provided  a  sounding  board  for  a 
short  report-out  on  the  Day  II  activities.  Jan  also  made  a  short,  informal  presentation  dealing  with 
the  positive  roles  that  the  FAA  sees  the  revised  CRM  advisory  circular  as  playing.  Mr.  Demuth 
clarified  some  of  the  issues  and  "realities"  as  to  the  current  non-use  of  CRM  in  the  qualification 
and  certification  of  airmen— and  the  difference  between  mandated  CRM  and  mandated  CRM 
evaluation,  especially  as  regards  to  developing  standards  which  are  not  currently  in  existence.  A 
lively,  intact  group  discussion  followed  on  CRM  assessment,  the  Draft  Advisory  on  CRM,  and  on 
different  uses  for  the  MPM.  A  synopsis  of  Mr.  Demuth’s  main  points  is  in  Appendix  E.  These 
are  significant  and  bear  reading,  as  they  relate  to  CRM  evaluation,  possible  future  NPRM,  and  the 
revised  CRM  Advisory  Circular. 


III.  RESULTS 

A.  Objective  I.  Steps  A  and  B 

General:  The  adequacy  and  accuracy  of  the  categories  and  clusters  within  which  the 
markers  are  grouped  was  questioned.  The  markers  are  the  building  blocks,  and  any  category  or 


"A  Small-Group  Oelphi  Paradigm'  (Lofaro,  R.J.,  1992).  Human  Factors  Society  Bulletin,  Vol.  35, 
No.  2;  'Exploratory  R&D:  Army  Aviation  Candidate  Classification  by  Specific  Helicopter*  (Lofaro,  R.J. 
and  Intano,  G.P.,  1989).  Proceedings  of  5th  International  Aviation  Psychologist  Symposium. 
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cluster  reorganization  will  cause  the  markers  either  to  migrate  or  to  be  altered.  Several  airlines 
indicated  that  they  view  the  markers,  as  now  written,  as  potential  research  tools.  However,  they 
also  indicated  that,  for  operational  analyses  and/or  to  capture  the  specific  performance  behaviors 
for  a  specific  airline  SOP,  the  markers  are  not  suitable.  The  markers  are  a  first  (and  important) 
step  toward  removing  some  ambiguity  about  flightcrew  CRM  and  Technical  performance,  and  are 
one  focus  that  can  be  used.  However,  they  have  definite  limitations;  for  example,  in  "conflict 
resolution",  the  markers  are  poor.  The  view  was  expressed  that  the  markers  do  not  apply 
uniformly  across  all  flightcrew  actions  and  decisions.  The  bottom  line  here  is  that:  (1)  the  existing 
markers,  categories  and  clusters  are  valuable  and  may  have  importance  as  a  research  tool;  (2)  the 
markers,  categories  and  clusters  must  be  open  to  change,  to  include  adding  new  ones  or  deleting 
existing  ones;  and,  (3)  any  set  of  markers,  categories  or  clusters  that  is  to  be  used  in  the 
operational  area— for  flightcrew  assessment,  LOFT  development,  curriculum  development-should 
be  developed,  analyzed  and  evaluated  by  an  airline  prior  to  use. 

The  5  point  Likert-like  scale  used  with  the  markers  was  also  critiqued.  This  scale  was  seen 
as  viable  //the  markers  were  used  as  a  research  tool.  For  operational  use,  a  "yes/no"  or  a  1-2-3 
type  assessment  scale  was  considered  superior.  However,  the  discussion  did  bring  out  that,  while 
it  may  be  possible  to  use  a  five-point  evaluation  scale,  there  was  considerable  disagreement  of  how 
to  (or,  if  there  even  was  a  reasonability  possibility  of)  differentiate  "3s";  "4s"  and  "5s"  on  such  a 
scale. 


Specific:  A  set  of  automation  management  markers  must  be  defined  and  a  rationale 
developed  for  where  they  go.  In  doing  this,  there  must  be  a  realization  that  this  marker  set  may  go 
into  two  (or  more)  places  and  that,  perhaps,  this  could  be  an  accurate  representation  of  operational 
reality. 

1.  A. 3  The  consensus  was  that  the  category  title,  "crew  self-critique"  carries  negative 
connotations.  The  preferred  term  (for  critique)  would  be  feedback;  this  implies  a  two- 
way  communication,  a  dialogue,  an  effort  at  improvement  using  the  feedback. 

2.  C.8;  1  through  7  This  set  of  markers  was  seen  as  redundant,  "wordy,"  and  has  some 
confusion  of  outcomes  and  performance. 

3.  C.7;  2  A  marker  must  be  developed  and  put  in  which  calls  out  "guard  against  routine- 
-engendered  complacency." 

NOTE:  At  this  point,  a  consensus  was  that  all  the  markers  could  be  critiqued,  at  a  great  cost  of 
time.  Since  the  general  discussion  of  the  markers  had  established  that  they  could  and  should  be 
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examined  and  altered,  it  was  decided  to  go  no  further  with  critiques  of  individual  markers.  In 
sum,  while  the  group  initially  did  not  feel  the  need  to  go  through  the  markers  one  by  one,  after 
some  four  specifics  were  brought  forward,  the  discussion  reverted  to  some  general  issues  after 
which  it  was  felt  further  specific  critique  was  not  called  for,  or  efficient. 

B.  Objective  II.  Part  1A 

Critique  and  Comments  on  Version  2.5  of  the  Draft  CRM  Advisory  Circular:  This 
discussion  was  a  strong  one  and,  in  some  part,  prompted  by  remarks  by  FAA  personnel  (at 
Northwest  Airline’s  "Industry  CRM  Workshop"  on  November  13,  1991,  in  Minneapolis)  that  a 
Notice  of  Proposed  Rule  Making  (NPRM)  on  CRM  was  foreseen  for  Spring  1992.  This 
engendered  new  interest  in  the  entire  draft  CRM  Advisory,  since  it  would  form  the  basis  for  the 
NPRM.  The  remark  also  evoked  strong  reactions  about  the  issues  involved  in  any  attempt  at  CRM 
evaluation. 

The  discussion  of  the  draft  Advisory  Circular  had  two  components.  In  the  first,  the  two 
figures  used  (Figures  1  and  2)  were  discussed.  In  the  second,  CRM  evaluation  was  thrashed  out. 

The  consensus  as  to  Figures  1  and  2  was  that  the  CRM  Integration  Model  presented  in 
these  two  figures  had  some  problems.  The  following  suggestions  and  concerns  were  brought  out: 

1.  A  better  articulated  rationale  for  Figure  2  is  needed. 

2.  Figure  2  should  be  used  without  the  categories/clusters.  These  categories/clusters  can 
be  placed  in  an  appendix.  Along  with  this  change,  put  the  crew  performance 
behavioral  markers  (CRM  and  Technical)  in  an  appendix. 

3.  The  content  validity  of  the  model  has  not  been  established. 

4.  Neither  Figure  1  nor  2,  nor  the  Advisory  Circular  itself  brought  out  the  importance  of 
criticality-rating,  allocation,  or  prioritization  of  skills  and  tasks  as  to  their  necessity, 
and  as  to  the  possible  changes  in  the  figures  which  can  result  by  doing  such  analysis. 

5.  Finally,  and  in  a  discussion  which  seemed  to  capture  all  of  the  above,  the  view  was 
brought  out  that  there  are  (many)  alternate  Figures  1  and  2  possible-dependent  on  an 
airline  SOPs,  analyses,  operational  emphases.  At  this  point,  Jon  Tovani,  of  Delta  Air 
Lines,  shared  with  the  group  an  alternate  to  Figure  2  which  had  been  developed  at 
Delta.  The  group  agreed  that  different  models  could  capture  CRM  Integration  and 
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that,  perhaps,  a  statement  to  that  effect  needed  to  be  in  the  CRM  advisory  or,  a  similar 
statement  that  Figure  1  and  2  show  gng  useful  way  of  conceptualizing  CRM 
Integration. 

The  advisory  also  should  encourage  each  airline  to  develop  a  CRM  Integration  model 
which  was  particularized  to  "fit"  its  goals  and  operations— and,  to  fit  the  CRM  Integration  Model 
into  the  issue  of  (eventual)  flightcrew  evaluation/assessment. 

NOTE:  There  is  now  (February  1992)  a  draft  revision  of  the  Advisory  Circular  that  obviates  some 
of  the  preceding  section.  However,  in  order  not  to  lose  some  historically  valuable  discussions  and 
consensus,  the  section  has  been  left  as  written  in  December  1991.  Appendix  C  contains  the  latest 
draft  of  the  advisory. 

CRM  Evaluation:  There  were  considerable  concerns  with  doing  this,  and  particular 
questions.  No  resolution  was  achieved  on  the  questions.  A  major  concern  was  a  feeling  that  CRM 
evaluation,  using  the  current  sets  of  CRM  and  Technical  Performance  markers  cannot  be  done- 
especially  if  such  an  evaluation  meant  using  each  and  every  marker,  for  each  and  every  evaluation. 
The  major  set  of  questions  revolved  around: 

1.  Are  CRM  skills  eventually  to  be  "bustable"?  (versus  Part  121,  Appendix  F) 

2.  Will  a  P.C./checkride  give  "equal"  weight  to  the  CRM  skills?  (A  view  was  expressed 
that  the  draft  CRM  Advisory  Circular  seemed  to  say  so). 

The  participants  concurred  that  the  industry  is  not  at  a  point  to  evaluate  CRM 
quantitatively.  All  of  the  participants,  except  one  air  carrier  training  officer,  thought  that 
qualitative  evaluation,  of  a  "SAT/UNSAT"  type,  may  be  possible.  A  concern  arises  in  giving  and 
justifying  a  CRM  "UNSAT"  if  all  technical  skills  are  rated  acceptable.  However,  this  consensus 
was  tempered  by  two  factors: 

1.  There  is  an  acceptance  that  now  is  the  time  for  the  airline  industry  to  think  about,  and 
begin,  efforts  in  CRM  assessments.  Some  air  carriers  are  already  involved  in  such 
efforts,  under  the  "no  jeopardy"  concept,  in  LOFT. 

2.  In  some  airlines  CRM  assessment  exists-with  SAT/UNSAT  based  on  three-point 
scales.  There  was  some  concern  that  what  one  airline  finds  acceptable  or  desirable  as 
assessment  might  be  forced  as  a  "standard"  on  other  air  carriers. 
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NOTE:  Several  participants  reported  that  their  airline,  at  present,  is  grading  CRM  indirectly  during 
a  check  flight.  Poor  CRM  performance  usually  results  in  poor  technical  performance.  However, 
the  UNSAT  is  attributed  to  poor  technical  performance  rather  than  CRM  shortfalls. 

C.  Objective  III 

This  objective  resulted  in  finding  six  potential  applications  for  the  MPM.  Caveats  made 
by  the  intact  group  were  that  the  MPM,  as  it  exists,  still  needs  further  development,  and 
particularization  to  each  airline’s  SOP’s  goals.  The  six  potential  uses  were: 

1.  Since  the  MPM  is  seen  to  be  generalizable  to  the  "expanded  crew"  concept,  it  has  use 
in  developing  and  integrating  CRM  for  maintenance,  flight  attendant  and  dispatch 
functions. 

2.  Since  the  MPM  demonstrates  and  "defines"  CRM  Integration  and  the  application  of 
CRM  skills  to  particular  flight  maneuvers,  the  MPM  lends  itself  to  LOFT  (or  other) 
scenario  development.  The  MPM  could  be  used  as  a  template  for  overlay  onto  an 
existing  scenario,  and/or  as  an  initiator  for  the  development  of  a  scenario.  In  these 
ways,  both  the  training  personnel  and  the  SME  pilots  involved  in  LOFT  development 
would  use  the  MPM. 

3.  The  MPM  could  be  used  in  a  rough  Front  End  Analysis  (FEA)  of  CRM  skills  to 
determine: 

a.  What  are  the  CRM  skills  necessary,  in  and  of  themselves,  for  a  safe  flight? 

b.  What,  in  toto,  goes  into  the  safe  movement  of  an  aircraft  from  point  A  to  point  B? 

4.  The  MPM  could  provide  a  focus  for  the  development/ delivery  of  Instructor  Evaluator 
training--"a  train  the  trainer"  tool. 

5.  Similar  to  III.C.3,  the  MPM  could  be  used  in  LOFT  development  to  calibrate  and/or 
evaluate  the  scenario(s). 

6.  Finally,  and  somewhat  akin  to  III.C.3,  the  MPM  could  be  used  to  identify  and 
demonstrate  what  CRM  skills  must  be  embedded  in  pilot  skill  training-and,  where  in 
the  training  cycle  this  should  occur. 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  13 


NOTE:  Jan  Demuth,  on  Day  III,  suggested  that  another  use  of  the  MPM  could  be  the  validation, 
at  the  sub-task  level,  of  any/all  of  the  crew  performance  markers.  He  also  suggested  that  such  a 
validation  could  occur  in  the  exercise  of  the  MPM. 

D.  Objective  II  (all  parts  except  1A) 

This  was  not  accomplished.  The  Discussion  Section  (below)  fully  explores  the  reasons  for 
this  lack.  However,  the  uses  of  the  MPM  are  being  explored  and  developed  at  United  Airlines, 
and  it  is  hoped  that  one  result  of  the  workshop  will  be  its  exploration,  particularization,  and  use  by 
other  air  carriers. 

IV.  SUMMARY  DISCUSSION 

The  workshop  plainly  did  not  accomplish  a  major  objective— Objective  n,  the  exercising, 
critiquing,  clarifying  and  finalizing  of  the  MPM.  One  of  the  reasons  for  this  has  been  mentioned 
prior,  that  the  MPM  was  not  completely  understood  or  fully  discussed  prior  to  the  workshop.  The 
major  reason  is  there  simply  was  not  sufficient  time  for  this  Objective,  even  if  all  of  Days  II  and 
III  could  have  been  allotted  to  it.  This  error  lies  with  the  designer/ facilitator-and  can  be  attributed 
to  his  aggressive  optimism  coupled  with  no  yardstick  by  which  to  estimate  this  task,  i.e.,  no  one 
had  attempted  any  of  the  components  of  Objective  II,  except  Kevin  Smith— and  Captain  Smith’s 
experiences  were  never  "timed."  However,  this  lack  of  opportunity  to  begin  to  deal  with  Objective 
II  was  offset,  to  some  degree,  by  the  best,  most  complete,  most  discussed  and  probed  dual 
presentations  on  the  MPM  that  had  yet  occurred.  The  presentations  by  Kevin  Smith  and  Bill 
Hamman  had  two  results: 

1.  While  the  MPM  had  been  gone  over  and  looked  at  by  others,  their  efforts  were  all-too- 
often  time-constrained;  colored  by  other,  similar  efforts  they  were  engaged  in  for  their 
various  airlines;  hampered  because  they  had  not  had  the  fullness  of  detail  and  the 
requisite  give-and-take  of  a  "complete"  question  and  answer  session  with  Kevin  Smith 
prior  to  the  workshop.  The  workshop  ameliorated  this,  to  some  degree. 

2.  The  presentations  at  the  workshop  made  the  various  participants  comfortable  enough 
with  their  knowledge  of  the  MPM  that  they  expressed  a  willingness  to  work  the  model, 
at  their  respective  airlines.  This  was  the  first  expression  of  such  an  intention/possible 
use  of  the  model  by  other  than  UAL.  There  is  the  possibility  of  a  future  workshop. 
Such  an  effort  could  be  coupled  with  agenda  time  at  ATA  working  group  sessions 
where  people  would  report  on  their  efforts,  problems  and  solutions  while  using  the 
MPM-or  some  modification  thereof. 
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In  retrospect,  it  was  probably  premature  to  have  committed  to  Objective  II.  However,  as  1992 
unfolds,  it  is  now  reasonable  to  expect  that  some  air  carriers  will  be  involved  in  on-going  work, 
and  the  reporting/sharing  of  it,  that  relates  to  CRM  Integration  and  the  MPM. 

The  workshop  plainly  did  accomplish  several  things--not  the  least  of  which  was  a  continuation 
of  the  non-critical,  open,  sharing  of  concerns,  issues  and  current  efforts  by  the  airlines  which 
participated.  The  Northwest,  one-day,  Industry  Workshop  (Minneapolis;  Nov.  14,  1991)  had  set  a 
tone  of  trust  through  the  non-judgmental  presentations  and  discussions  in  CRM  concerns,  issues, 
efforts  by  the  major  carriers  with  the  FAA  and  NTSB  in  attendance.  That  is  rare.  The  workshop, 
in  some  ways,  built  on  and  continued  this  Northwest-hosted  effort. 

The  workshop  also  demonstrated  that  there  is  diversity  within  a  shared  goal  and  some  shared 
tools.  Each  airline  indicated  that  they  were  looking  at  using,  and  modifying  for  their  operations 
and  in  light  of  their  analyses,  the  behavioral  markers  and  the  CRM/Technical  schemes  laid  out  in 
the  draft  Advisory  Circular.  Further,  some  of  the  criticisms  and  modifications  which  came  up  at 
the  workshop  not  only  made  sense,  but  also  will  become  part  of  current  and  future  airline  work  by 
virtue  of  being  expressed,  shared  and  being  a  part  of  this  report. 


V.  RECOMMENDATIONS 

NOTE:  These  recommendations  are  presented  unchanged  as  received  from  the  participants 
following  their  review  of  the  draft  of  this  report. 

1.  Work  should  continue  on  CRM  Integration  and  the  MPM. 

2.  Another  workshop  of  approximately  the  same  length  should  take  place  during  the  first 
half  of  1992  to  continue  work  in  this  area.  The  goal  of  this  new  workshop  should  be  to 
complete  Objective  II. 

a.  Workshop  participants  should  be  polled  to  determine  whether  or  not  another 
workshop  is  required  or  desired. 

3.  Evaluate  the  MPM  and  other  alternatives,  thus: 

Establish  a  specific  task  for  each  airline  to  use  with  the  model  or  representative. 
Examples  of  tasks  could  be  specific  as  a  potential  for  windshear  on  takeoff  or  could  be 
a  complete  LOFT  from  A  to  B.  Have  each  airline  use  their  SME  group  to  use  the 
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CRM  integration  model.  After  this  is  completed,  a  joint  conference  should  be  held  to 
compare  the  key  factors  of  CRM  for  this  particular  task.  These  results  would  then  be 
analyzed  to  validate  the  sensitivity,  usefulness  and  accuracy  of  the  model  to  establish 
the  CRM  elements  of  aircraft  operations.  This  type  of  approach  would  also  add 
valuable  information  to  assist  other  subcommittees  of  the  ATA,  i.e.,  LOFT  subgroup. 

4.  A  formal  acknowledgment  is  made  of  the  efforts/time  spent  by  Kevin  Smith  in  the 
creating  and  developing  of  the  MPM  for  the  good  of  the  Aviation  Industry. 
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VI.  ALTERNATE  VIEWPOINT 

One  air  carrier  indicated,  in  their  review  of  the  draft  of  this  report,  that  they  wished  to 
express  certain  reservations  and/or  disagreements  with  portions  of  the  draft  report.  In  order  to 
make  this  report  as  complete  and  comprehensive  as  possible,  their  comments  are  presented  below: 

A.  1.  The  MPM  is  seen  as  a  type  of  task  analysis  tool  that,  to  its  credit,  focuses  on  both 

technical  and  interactive  components  of  crew  performance. 

2.  The  MPM  seems  to  validate  what  has  been  explained  by  Helmreich’s  behavioral 
markers  and  other  airlines’  own  ongoing  task  analysis  for  AQP. 

B.  The  MPM  implies  an  evaluative  strategy  that  is  arbitrary  and  incomplete: 

1 .  The  MPM  is  arbitrary  in  its  assignment  of  CRM  skills  to  specific  tasks  in  a  scenario. 

2.  The  MPM  is  incomplete  in  that  it  lacks  specific  criteria  for  a  scaled  grading  scheme. 

C.  Continuing  work  on  the  MPM  should  focus  on  empirically  validating  the  assignment  of 
behavioral  markers  to  tasks.  This  validation  could  use  data  from  actual  crew  performance 
observations;  e.g.,  LOFT  observations. 
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APPENDIX  A: 


Edited  Version  of  Phase  I  Materials 
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I.  CREW  RESOURCE  MANAGEMENT  (CRM) 

A.  History 

During  the  late  1970s  and  through  much  of  the  ’80s,  major  air  carriers  experienced 
a  number  of  fatal  accidents  where  the  causal  factors  were  not  part  malfunctions,  flight 
control  errors,  lack  of  skills,  incorrect  procedures,  or  any  of  the  traditional  aviation  accident 
categories.  Rather,  there  were  causal  chains  of  events/  actions/inactions  where  the  common 
threads  were  communication  problems  and  a  lack  of  cohesive,  crew-level  planning  and 
decision-making.  From  many  accident  (and  operations)  analytic  studies  came  the  realization 
that  a  new  element  was  necessary  in  flightcrew  preparation.  As  human  factors  experts 
began  to  look  more  and  more  at  the  group/interactive  aspects  of  flightcrew  functioning,  it 
became  apparent  that  many  accidents  and  incidents  could  be  attributed  to  a  lack  of  CRM- 
type  training  and  skills;  that  human  performance  errors  were  linked,  on  the  flightdeck,  to 
factors  that  had  previously  not  been  recognized,  identified  or  investigated.  Both  United  and 
KLM  took  the  lead  in  developing  what,  at  that  time,  was  termed  Cockpit  Resource 
Management  training. 

Dr.  John  Lauber  and  Dr.  Clay  Foushee  of  NASA-Ames  led  the  way  in  bringing 
CRM  to  the  attention  of  the  aviation  community.  Along  with  Dr.  Robert  Heimreich  of  the 
University  of  Texas  they  began  to  hammer  out  the  basic  CRM  concepts  and  theory  while 
doing  research  on  crew  performance  and  crew  skills. 
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The  late  ’80s  brought  some  measure  of  agreement  in  aviation  that  CRM  training 
had  the  potential  to  affect  safety  of  flight  and  crew  performance  positively.  The  FAA,  in 
recognition  of  this,  would  grant  a  waiver  on  recurrency,  if  the  airline  used  CRM  Training. 

A  thorny  issue,  officially  recognized  at  a  major  CRM  workshop  (L986),  was  the 
evaluation  of  CRM— the  issues  were  many  and  included  not  only  how,  but  also  why,  and 
what  criteria  were  to  be  used,  as  well  as  the  individual  in  a  crew  versus  the  crew  level  of 
performance.  There  are  still  issues  to  be  resolved  and  there  are  still  disagreements. 
However,  Dr.  Helmreich,  in  a  joint  NASA/FAA  project,  and  others  continue  to  research 
the  issues  and  to  develop  new  training,  as  well  as  evaluation  methods.  These  include  the 
Line/LOFT  Checklist  (LLC);  CRM  training  participant  surveys;  CRM  participant  attitudes 
and  personality  factors  measures  as  related  to  performance  differences  across/within  airlines 
and  their  fleets. 

The  next  major  shift  came  in  the  very  late  1980s  and  early  1990s.  This  was 
occasioned  by  an  FAA  effort  to  establish  a  Federal  Air  Regulation  (FAR)  which  would 
allow  and  encourage  airlines  to  develop  their  own  alternative  training  programs/flightcrew 
training  programs  which  followed  certain  FAA  guidelines,  but  whose  establishment  was 
voluntary;  programs  which  led  to  alternative  methods  of  airman  qualification,  certification 
and  recurrency.  This  regulation  (SFAR  58),  published  in  October  1990,  is  called  the 
Advanced  Qualification  Program.  In  any  innovative  training  program  that  an  air  carrier 
proposes  under  this  SFAR,  CRM  training  is  mandated  and  Line  Oriented  Flight 
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Training/Line  Oriented  Simulation  (LOFT/LOS)  is  encouraged.  This  is  in  line  with  the  fact 
that  many  existing  air  carrier  CRM  programs  do  include  a  flight  simulator  scenario  (a 
LOFT)  to  be  critiqued  as  a  "final  exercise". 

This  SFAR,  while  mandating  CRM  in  any  Advanced  Qualifications  Program  (AQP) 
curriculum,  also  mandated  that  it  be  evaluated— and  evaluated  in  a  way  to  demonstrate 
proficiency.  In  fact,  all  AQP’s  must  be  proficiency-based,  and  the  proficiencies  must  be 
validated.  At  this  point,  the  prior  issues  of  CRM  evaluation  had  to  be  resolved.  A  first 
step  was  the  development,  by  Bob  Helmreich,  of  Behavioral  Markers;  behaviors  which 
needed  to  be  present  as  indicators  of  the  possession  of  the  major  CRM  component  skills. 
These  markers  also  had  5  point  evaluation  scales  to  measure  the  skill-level  of  each  CRM 
component.  These  are  called  "Crew  Performance  Markers"  and  have  been  expanded  by 
Kevin  Smith  to  include  the  Technical/Flight  Control  components. 

All  of  this  has  lead  to  the  initial  work,  by  Bob  Helmreich,  Kevin  Smith  and  Jan 
Demuth,  on  a  CRM  integration  model.  This  model  integrates  the  stick  and  rudder 
skills/knowledge  needed  for  flight  control  with  CRM  skills  and  knowledge.  The  model,  to 
be  covered  more  fully  in  Section  II,  is  called  the  (Flightcrew)  Mission  Performance  Model 
(MPM).  It  focusses  on  the  integrated  crew  functions  necessary  and  critical  for  safe  and 
effective  completion  of  any  flight. 
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B.  Phase  I  and  II  Purpose 

The  mission  performance  model  is  in  its  initial  stages  of  development.  What  needs 
to  be  done  is  to  complete  it,  i.e.,  add  to  it,  exercise  it,  modify  it,  "finalize"  it.  In  doing 
this,  we  see  a  series  of  steps,  or  objectives: 

1 .  Apply  existing  form  of  the  model  for  familiarity 

2.  Ascertain  what  needs  to  be  added/subtracted/changed 

3.  Re-apply  model-over  different  phases  of  flight— and  ensure  that  it  can  handle  all 
conditions  (e.g.,  emergencies)  and  procedures 

4.  Ascertain  if  the  model  can  deal  differentially  with  different  fleets  and  company 
SOP’s 

5.  "Find"  all  algorithms  or  rules  embedded  in  the  model 

6.  Identify/develop  uses  for  the  modeL-emphasizing  LOS  design  and  evaluation 
and  aircrew  performance  behaviors 

What  we  have  given  you  here  is  obviously  a  broad-brush  look.  There  will  be  a 
more  specific  version  of  these  (working)  objectives  given  to  you  at  the  Phase  II  workshop. 
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II.  THE  MISSION  PERFORMANCE  MODEL 
A.  Overview 

The  purpose  here  is  not  to  make  you  an  expert  in  the  model— it  is  still 
developmental— but,  rather,  to  acquaint  you  with  its  rationale  and  provide  a  few  examples. 
In  this  way,  we  hope  that  you  will  come  to  our  Special  Team  Workshop  with  definite 
reactions  and  ideas.  The  model  is  based  on  the  concepts  that: 

1.  Flying  is  an  integrated,  mission-oriented  activity  which  must  be  evaluated  as 
such. 

2.  The  crew’s  performance  is  not  adequately  captured  by  totaling  the  sum  of  the 
component  tasks/sub-tasks/elements.  The  focus  must  be  on  crew  function- 
usually  at  the  task  and  critical  sub-task  levels. 

3.  Flight  proficiency  skills/knowledge  are  interwoven,  interdependent,  and 
necessarily  interact  with  the  CRM  skills/knowledge  differentially  across  tasks 
and  conditions.  These  interactions  can  be  specified  by  a  matrix-type  crew 
mission  performance  model  using  the  tasks  which  comprise  a  mission/leg. 

4.  The  model  can  capture  these  interactions;  can  be  sensitive  to  changes  in  both 
task  and  mission,  i.e.,  show  that,  for  different  tasks  and  conditions,  both  the 
technical  (flight]  proficiency  skills  and  the  CRM  skills,  and  their  interactions, 
will  vary.  This  is  an  indication  that  the  model  has  a  measure  of  discriminatory 
power,  or  "sensitivity"  to  changes  in  task  and  conditions. 
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5.  Helmreich’s  "Behavioral  Markers"  can  adequately  delineate  CRM  skills  and 
provide  a  basis  for  the  CRM-specific  Flightcrew  Mission  Performance  Model. 

6.  The  bases  for  the  technical  proficiency  evaluation  currently  exist,  in  a 
behavioral  marker-type  format,  and  Scales  exist,  or  can  be  developed,  for 
evaluation  of  all  these  proficiencies. 

B.  Discussion  and  Example  of  MPM 

The  Mission  Performance  Model  has  embedded  within  it  the  concept  of  functions. 

It  is  proposed  that  the  model,  as  constructed,  represents  all  significant  functions  necessary 
for  the  successful  completion  of  an  air  transport  mission. 

This  model  views  crew  performance  as  consisting  of  system  level  functions  which 
represent  the  mechanisms  that  are  used  to  perform  a  mission  activity.  The  importance  of  a 
model  that  is  founded  on  a  set  of  systems  level  functions  cannot  be  overstated.  Moreover, 
the  model  delineates  crew  performance  at  a  level  of  abstraction  that  is  significantly  different 
than  the  current  descriptions  of  individual  performance. 

For  individuals,  performance  has  often  been  seen  as  a  series  of  discrete  tasks, 
where  each  task  was  further  decomposed  to  reveal  a  set  of  subtasks  combined  with  the 
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requisite  knowledge  and  skills  necessary  for  subtask  completion.  For  many  applications, 
such  as  aircrew  training,  this  produces  a  large  collection  of  task,  knowledge  and  skill  data. 

In  most  traditional  pilot  or  crew  training  programs,  these  are  taught  individually  as 
isolated  knowledge  components.  Consequently,  the  trainee  is  left  with  the  responsibility  of 
combining  these  isolated  knowledge  components  into  integrated  wholes  (Merrill,  1989). 

The  linear  decomposition  of  individual  tasks  does  not  address  integrated  functioning 
nor  does  it  reveal  how  tightly  coupled  teams  (flight  crews)  perform,  thus  an  analytical 
process  other  than  the  traditional  task  analysis  approach  is  considered  necessary.  We, 
therefore,  have  chosen  the  Functional  Modeling  approach. 

The  MPM  consists  of  a  set  of  functions  that  can  be  activated  by  inserting  an 
instance/example;  in  other  words,  asking  the  function  to  specify /describe  a  particular 
activity  or  situation  in  the  mission.  If  a  particular  function,  e.g.  "Workload  Management," 
was  asked  to  "spin  out"  the  components  of  a  particular  mission  activity,  such  as  take  off 
with  an  engine  failure  at  Vj,  then  the  function  should  be  able  to  organize,  sequence, 
distribute,  and  coordinate  key  crew  actions  so  that  a  successful  outcome  could  be  assured. 
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This  workload  management  function,  then,  can  be  viewed  as  a  generic  performance 
statement  that 

1.  Can  be  applied  to  many  mission  activities/situations,  and 

2.  Can  be  activated  for  the  application  to,  and  specification  of,  any  one  of  these 
activities/situations . 

(Note:  Any  case  in  which  the  Mission  Performance  Model  is  activated  in  order  to 
accomplish  a  particular  mission  activity  shall  be  termed  "instantiation."  Instantiation  means 
the  function  has  been  activated,  or  made  particular,  by  using  values/activities  in  place  of 
variables.] 

To  return  to  our  previous  example  of  workload  management  during  a  take  off 
activity  with  engine  failure  at  Vj,  we  see  that: 

(1)  The  workload  function  is  being  asked  to  deal  with  the  "effective" 
management  of  workload  during  this  critical  flight  maneuver. 

(2)  Once  targeted,  the  specifics  of  the  critical  flight  maneuver  are  embedded 


within  the  function. 
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(3)  The  evaluation  of  this  function,  with  respect  to  the  demands  being  placed  on 
it  during  this  critical  flight  maneuver,  gives  us  insight  into  the  robustness  of 
the  function. 

We  have  been  using  the  term  "Mission  Performance  Model"  up  to  this  point,  and 
before  we  proceed  with  "CRM  Integration,"  a  few  words  about  the  concept  of  a  model  are 
in  order. 

The  Mission  Performance  Model  is  one  which  can  specify  the  components  of 
flightcrew  "effectiveness"  (effective  performance).  That  the  model  represents  effectiveness 
is  important  to  understand  since,  if  the  crew  is  really  engaging  in  the  set  of  functions  that 
are  germane/linked  to  the  problem  at  hand,  and  if  these  functions  are  the  prerequisites  for  a 
successful  outcome,  then  effectiveness  has  been  demonstrated. 

Similarly,  the  model  is  prescriptive-it  prescribes  what  needs  to  be  accomplished  for 
the  crew  to  perform  effectively.  For  example,  we  can  specify,  during  the  LOFT  design 
process,  what  are  very  likely  to  be  the  necessary  crew  behaviors. 
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C.  The  Concept  of  CRM  Integration 

The  first  step  in  the  analytic  process  of  "CRM  Integration"  is  to  note  that  for  a 
particular  mission  activity,  an  appropriate  set  of  technical  and  human  factor  functions  will 
need  to  be  performed,  and  will  need  to  be  performed  together.  For  example,  if  one  were  to 
focus  on  the  vertical  navigation  components  of  the  initial  approach  phase  of  a  mission,  both 
the  workload  management  and  situation  assessment  functions  come  into  play  in  conjunction 
with  the  technical-based  vertical  navigation  function.  Further,  all  three  of  these  functions 
can  be  shown  to  interact.  This  notion  of  interaction  will  be  explored  later,  but,  for  now, 
we  want  to  show  that  very  often  we  will  have  both  human  factors  and  technical  functions 
working  together  to  perform  a  particular  mission  activity. 

D.  The  Demonstration  of  CRM  Integration 

The  CRM  integration  process  is  demonstrated  by  creating  a  matrix  array  where  the 
left  column  is  any  mission  activity  or  situation  (i.e.,  the  flight  tasks  involved),  and  the  top 
row  of  the  matrix  specifies  both  the  CRM  and  flight  Performance  Markers  in  the 
description  of  the  Mission  Performance  Model  (MPM).  (These  markers  can  be  found  in  the 
Draft  FAA  CRM  Advisory  Circular,  Appendix  D.]  Once  the  matrix  is  set  up,  relationships 
between  various  activities  and  functions  can  be  identified.  These  are  shown  in  the 
row/column  intersections.  These  intersections  are  where  the  functions  come  into  play  to 
accomplish  a  particular  flight  task. 
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There  are  three  large  clusters  under  which  the  Performance  markers  for  the 
Cognitive  and  Interpersonal  (C&I)  CRM  are  gathered: 

1.  Team  Formation  and  Management  Tasks  (Cluster  A); 

2.  Communication  and  Decision  Tasks  (Cluster  B);  and, 

3.  Situational  Awareness  and  Workload  Management  Tasks  (Cluster  C). 

Each  of  these  clusters  is  further  broken  out  into  categories,  w’hich  comprise  the 
critical  tasks  that  are  performed  in  each  cluster.  These  categories  are  given  single  or 
double  digit  numbers  as  identifiers— such  as,  under  the  Communication  Process  and 
Decision  Tasks  Cluster,  we  will  find  four  categories,  e.g.,  Briefing,  Crew  Self-Critique. 
The  categories,  under  each  cluster,  are  finally  broken  out  into  descriptive  "Performance 
markers."  These  Performance  markers  delineate  the  actions  and  behaviors  which  make  up 
a  category.  There  are  usually  two  to  seven  Performance  markers  (for  example,  "Identifies 
potential  problems,  such  as  WX,  delays  and  abnormal  systems  ops,"  or  "sets  expectations 
for  how  deviations  from  SOP  are  to  be  handled")  defined  for  each  category. 

Further,  each  Performance  marker,  in  each  category,  in  each  Cluster,  then  is 
evaluated  on  a  five-point,  performance-anchored,  scale— from  "poor  performance, 
significantly  below  expectations,"  through  "exceptional  performance,  significantly  above 


standard. 
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Using  an  operational-type  analysis,  the  CRM  and  flight  skills  are  identified,  as  to 
which  skills  are  brought  into  play  across  the  crew’s  functioning.  The  matrix  is  thus 
completed  (see  Figure  1). 
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Several  things  must  be  noted:  Not  every  row/column  intersection  in  the  matrix  will 


have  either  CRM  and/or  flight  skills  in  it. 
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Not  every  row/column  intersection  which  has  flight  skills  will  have  CRM  skills — 
and  vice-versa.  However,  each  time  a  row/column  intersection  has  both  flight  control  and 
CRM  skills,  then  that  is  an  indication  of  the  fact  that  both  sets  of  these  skills  must  be 
interrelated/integrated  in  order  to  perform  that  task/sub-task—and  the  mission  itself-- 
successfully. 

As  was  said,  the  Helmreich  Performance  Markers  and  their  decomposition  are 
included.  Some  examples,  from  Figure  1,  of  the  "shorthand"  that  the  model  uses  is  as 
follows: 

D  1  means  Cluster  D,  Category  1 
and 

C  810  means  Cluster  C,  Category  8,  Marker  10 

The  mission  sub-tasks  are  also  further  broken  out/decomposed  into  smaller 
elements.  For  example: 

"6. 1.1.2"  means  (left-to-right) 

Task  6.1,  with  sub-task  .1  and  element  .2 

Task  6.1  is  to  perform  an  initial  approach.  Sub-task  .1  (i.e.,  6.1.1)  of  it  is  to 
assess  environmental  factors,  and  6. 1.1.2  represents  element  .2  of  subtask  .1  of  task  6.1. 
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The  tasks,  subtasks,  and  elements  came  from  an  existing  United 
Airlines  task  analysis  which  is  not  included  due  to  its  size. 


The  criticality  and  strength  of  the  interrelationships  between  the  CRM  performance 
markers  and  the  flight  control/technical  skills  may  be  able  to  be  measured  and  evaluated 
(see  Figure  2).  During  Phase  II,  we  will  try  to  develop  and  refine  techniques  and  scales  for 
any  interactions. 
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III.  THE  PHASE  II  WORKSHOP  ("SPECIAL  TEAM") 

A.  Objectives 

A  general  introduction  to  these,  and  a  general  statement  of  them,  has  been 
presented  in  Section  II.  As  the  workshop  progresses,  things  will  arise  (or  fall  away)  which 
can  change  any  prior,  stated  set  of  objectives.  The  modified  Delphi  format  that  will  be 
used  should  provide  the  framework  for  achieving  these  goals— in  a  relatively  short  time  and 
with  quality  results. 

On  Day  I  of  the  workshop,  we  will  present  a  specific,  sequenced  set  of 
objectives-for  discussion,  clarification,  modification,  agreement— and, 
completion. 

B.  Format 

We  will  form  two  work  teams.  Each  team  will  work  as  individuals  on  some 
iterations  and  as  a  team  on  some  iterations-but,  always  as  a  team  for  finalizing 
data/achieving  consensus.  Additionally,  the  entire  workshop  will  function  as  a  "super¬ 
team",  meeting  to  integrate  and  finalize  the  data  from  the  two  work  teams. 

The  workshop  will  be  facilitated  by  Dr.  Ron  Lofaro  who  has  designed  and  used  a 
highly  modified  Delphi  process  to  develop  operational  aviation  systems  on  two  other 


occasions. 
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There  may  be  some  brief  training  and  discussion  on  group  function,  consensus  and 
"group-think". 

There  will  be  a  complete  and  comprehensive  set  of  protocols  and  reference 
materials  provided— and  discussed— prior  to  any  data  being  generated. 

The  Workshop  flow-chart,  as  to  the  model: 


i - 

v 

USE  IT 


►  MOD/FINALIZE  IT 


A 

J 


►  VALIDATE  IT* 
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Phase  II  Workshop 


Time:  0930  on  19  Nov.  to  1600  on  21  Nov. 

Place:  Washington,  DC 


Participant  List:  Mr.  Jan  Demuth,  FAA 

Mr.  Doug  Farrow,  FAA 

Dr.  Ron  Lofaro,  FAA 

Capt.  Fred  Lorenz,  The  Boeing  Company 

Capt.  Frank  Tullo,  Continental  Airlines 

Capt.  Jon  Tovani,  Delta  Air  Lines 

Capt.  Ted  Mallory,  Northwest  Airlines 

Capt.  Stan  Smartt,  Northwest  Airlines 

Capt.  Kevin  Smith,  United  Airlines 

Capt.  Bill  Hamman,  M.D.,  United  Air  Lines 

LTC  Mike  Jobanek,  USAFR/MiTech,  Inc. 

Dr.  Tony  Ciavarelli,  USNPGS 
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IV.  HOMEWORK  ASSIGNMENT 

1.  Please  read  this  package  carefully. 

2.  Read  over  the  performance  markers,  both  CRM  and  technical.  Critique  these 
as  to  completeness  and  correctness;  write  down  what  you  feel  is  missing  or 
needs  to  be  changed.  Please  bring  your  notes  when  you  come  to  the  Workshop 
(refer  to  them  as  Objective  1A). 

3.  Read  over  the  performance-anchored  rating  scales  used  for  the  performance 
markers.  As  in  1.  above,  critique  these  and  bring  these  notes  with  you  also 
(refer  to  them  Objective  IB). 

4.  Consider  the  Mission  Performance  Model.  How  do  you  feel  it  can  be  used  best 
(e.g.,  in  aircrew  training?  LOFT  evaluations?  check  airmen?)?  And,  where  do 
you  feel  it  can  have  the  least,  or  no,  value?  In  doing  this,  refer  to  the  most 
current  draft  of  the  CRM  Advisory  Circular.  Again,  whatever  notes  you  make 
here,  please  bring  them  to  the  Workshop  (refer  to  them  as  Objective  III). 

5.  Please  fill  out  the  Participant  Data  Sheet  (Appendix  C)  and  mail  to: 

Dr.  Ronald  J.  Lofaro 
ATTN:  ARD-200 
Federal  Aviation  Administration 
800  Independence  Avenue,  S.W. 

Washington,  D.C.  20591 
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APPENDIX  A  OF  THE  PHASE  I  PACKAGE 
CREW  RESOURCE  MANAGEMENT 
(CRM) 

ADVISORY  CIRCULAR  120-51 
Draft  2.4 


I In  the  Final  Report  on  the  Workshop,  this  has  been 
replaced  with  Appendix  C,  the  most  recent  draft 
of  the  Advisory  Circular.] 
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APPENDIX  B  OF  THE  PHASE  I  PACKAGE 


CREW  PERFORMANCE  MARKERS: 

HUMAN  and  TECHNICAL  FACTORS 
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CREW  PERFORMANCE  MARKERS 

Crew  Performance  Markers  are  conceptualized  as  a  way  to  understand  operational 
quality  standards  that  are  keys  to  an  efficient,  safe  and  successful  operation. 

Crew  performance  markers  in  effect  provide  for  a  "model"  of  effective  crew 
performance.  This  "performance  model",  once  specified,  can  be  used  to  1)  construct 
meaningful  line  oriented  flight  training  sessions  and  2)  evaluate  crew  performance  in  a 
way  that  provides  for  meaningful  feedback. 

However,  before  either  training  program  development  or  crew  performance  assessment 
can  occur,  a  conceptual  understanding  of  crew  performance  markers  is  considered 
necessary.  They  are  separated  into  human  factors  and  technical  related  items.* 

I.  Crew  Performance  Markers--Human  Factors 

A.  Communications  Processes  and  Decision  Behavior 

Briefing  (conduct  and  quality).  The  effective  briefing  will  be  operationally  thorough, 
interesting,  and  will  address  coordination,  planning,  and  problems.  (Although  primarily  a 
Captain  responsibility,  other  crewmembers  may  add  significantly  to  planning  and 
definition  of  potential  problem  areas.) 

Inquiry/Advocacy/Assertion.  This  rating  assesses  the  extent  to  which  crewmembers 
advocate  the  course  of  action  they  feel  best,  even  when  it  involves  conflict  and 
disagreements  with  others. 

Crew  self-critique  (decisions  and  actions).  This  item  evaluates  the  extent  to  which 
crewmembers,  conduct  and  participate  in  a  debriefing,  operational  review,  and  critique 
of  activities,  which  includes  the  product,  the  process,  and  the  people  involved.  Critique 
can,  and  should,  occur  during  an  activity,  and/or  after  completion  of  the  activity. 

Conflict  resolution.  If  crewmembers  engage  in  conflict  while  attempting  to  decide  on  a 
course  of  action  or  for  any  other  reason,  the  effectiveness  of  means  used  to  resolve  the 
conflict  and  the  use  of  available  resources  is  rated. 


*  Bold,  italicized  markers  apply  to  Advance  Technology  Flightdecks. 
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Communications/Decisions.  This  rating  reflects  the  extent  to  which  free  and  open 
communication  is  practiced.  It  includes  providing  necessary  information  at  the 
appropriate  time  (for  example,  initiating  checklists,  alerting  others  to  developing 
problems).  Active  participation  in  decision-making  process  encouraged  and  practiced. 
Questioning  of  actions  and  decisions  is  proper.  Decisions  made  are  clearly 
communicated  and  acknowledged. 

1 .  Establishes  team  concept  and  environment  for  open/interactive  communications 
(e.g.,  calls  for  questions  or  comments,  answers  questions  directly,  listens  with 
patience,  does  not  interrupt  or  "talk  over",  does  not  rush  through  the  briefing, 
makes  eye  contact  as  appropriate). 

2.  Identifies  potential  problems  such  as  weather,  delays,  and  abnormal  system 
operations.  Sets  expectations  for  how  deviations  from  S.O.P.  are  to  be  handled. 

3.  Provides  guidelines  for  crew  actions  --  division  of  labor  and  crew  workload 
addressed. 

4.  Includes  cabin  crew  as  part  of  team  in  the  briefing,  as  appropriate. 

5.  Operational  decisions  are  clearly  stated  to  other  crewmembers  and  acknowledged. 

6.  "Bottom  lines"  are  established  and  communicated  for  safety  of  operations.  The 
"big  picture"  and  the  game  plan  are  shared  within  the  team  including  flight 
attendants  and  others. 

7.  Crewmembers  are  encouraged  to  state  their  own  ideas,  opinions,  and 
recommendations. 

8.  Crewmembers  speak  up,  and  state  their  information  with  appropriate  persistence, 
until  there  is  some  clear  resolution  and  decision. 

9.  Crewmembers  are  encouraged  to  ask  questions  regarding  crew  actions  and 
decisions,  and  answers  are  provided  openly  and  non-defensively. 

10.  Critique  is  given  at  appropriate  times,  both  low  and  high  workload,  and  is  made  a 
positive  learning  experience  for  the  whole  crew-feedback  is  specific,  objective, 
based  on  observable  behavior,  and  given  constructively. 
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1 1 .  Critique  is  accepted  objectively  and  non-defensively,  deals  with  positive  as  well  as 
negative  aspects  of  crew  performance. 

12.  When  conflicts  arise,  the  crew  remains  focused  on  the  problem  or  situation  at  hand. 
Crewmembers  listen  actively  to  ideas  and  opinions  and  admit  mistakes  when 
wrong. 

13.  Assignment  of  blame  is  avoided  --  the  focus  is  on  determining  what  is  right,  not 
who  is  right.  Crewmembers  treated  with  empathy  and  respect.  When  there  is 
time,  crewmembers  explain  "why"  particular  decisions  were  made. 

14.  Establishes  policy  guidelines  for  the  operation  of  automated  systems  (i.e.  when 
system  will  be  disabled,  programming  actions  that  must  be  verbalized  and 
ackno  w/edged). 

1 5.  Specifies  PF  and  PNF  duties  and  responsibilities  with  regard  to  automated  systems. 

1 6.  Crewmembers  verbalize  and  acknowledge  entries  and  changes  to  automated 
systems. 

1 7.  Crewmembers  question  status  and  programming  of  automated  systems  to  verify 
and  ensure  situational  awareness. 


B.  Team  Building  and  Maintenance 

Leadership,  Followership,  and  Concern  for  Tasks.  This  rating  evaluates  the  extent  to 
which  appropriate  leadership  and  followership  is  practiced.  It  reflects  the  extent  to 
which  the  crew  is  concerned  with  the  effective  accomplishment  of  necessary  tasks. 

Interpersonal  Relationships/Group  Climate.  This  evaluation  reflects  the  quality  of 
observed  interpersonal  relationships  among,  and  the  overall  climate  of,  the  flightdeck. 
This  is  independent  of  demonstrated  concern  with  accomplishment  of  required  tasks. 

1 .  Coordinates  flightdeck  activities  to  establish  proper  balance  between  authority  and 
assertiveness;  acts  decisively  when  the  situation  requires. 

2.  Demonstrates  desire  to  achieve  most  effective  possible  operation 
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3.  Ensures  that  group  climate  is  appropriate  to  operational  situation  (i.e.  social 
conversation  in  low  workload  conditions  but  not  high). 

4.  Shows  sensitivity  and  ability  to  adapt  to  other  crewmembers'  personalities  and 
personal  characteristics. 

5.  Recognizes  symptoms  of  psychological  stress  and  fatigue  in  self  and  others  (e.g., 
note  when  a  crewmember  is  not  communicating,  and  draw  him/her  back  into  the 
team;  recognize  when  they  are  experiencing  "tunnel  vision,"  and  seek  help  from  the 
team). 

6.  "Tone"  in  the  cockpit  is  friendly,  relaxed,  supportive. 

7.  Ensures  that  non-operational  factors  such  as  social  interaction  do  not  interfere  with 
necessary  task  duties. 

8.  During  times  of  low  communication,  crewmembers  check  in  with  each  other  to  see 
how  they  are  doing. 

9.  Recognizes  and  deals  with  demands  and  resources  posed  by  operation  of  automated 
systems. 

1 0.  Reverts  to  lower  level  of  automation  when  programming  demands  could  reduce 
situational  awareness  or  create  work  overloads. 


C.  Workload  Management  and  Situational  Awareness 

Preparation/Planning/Vigilance.  This  rating  indicates  the  extent  to  which  crews 
anticipate  contingencies  and  actions  that  may  be  required.  Excellent  crews  are  always 
"ahead  of  the  curve"  while  poor  crews  continually  play  catch  up.  Vigilant  crews  devote 
appropriate  attention  to  required  tasks  and  respond  immediately  to  new  information.  A 
crew  indulging  in  casual  social  conversation  during  periods  of  low  workload  is  not 
necessarily  lacking  in  vigilance  if  flight  duties  are  being  discharged  properly. 

Workload  Distribution/Distraction  Avoidance.  This  is  a  rating  of  time  and  workload 
management.  It  reflects  how  well  the  crew  managed  to  distribute  the  tasks  and  avoid 
overloading  individuals.  It  also  considers  the  ability  of  the  crew  to  avoid  being 
distracted  from  essential  activities  and  how  work  is  prioritized. 
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1 .  Actively  monitors  weather,  aircraft  systems,  instruments  and  ATC  communications, 
sharing  relevant  information  with  the  rest  of  the  crew. 

2.  Avoids  "tunnel  vision,"  being  aware  of  factors  such  as  stress  that  can  reduce 
vigilance— thus,  monitoring  the  performance  of  other  crew  members. 

3.  Stays  "ahead  of  curve"  in  preparing  for  expected  or  contingency  situations 
(including  approaches,  weather,  etc.) 

4.  Verbally  insures  that  cockpit  and  cabin  crew  are  aware  of  plans. 

.  Workload  distribution  is  clearly  communicated  and  acknowledged  to  maximize 
efficiency. 

6.  Ensures  that  secondary  operational  tasks  (i.e.  dealing  with  passenger  needs, 
company  communications)  are  prioritized  so  as  to  allow  sufficient  resources  for 
dealing  effectively  with  primary  flight  duties. 

7.  Recognizes  and  reports  overloads  in  self  and  others. 

8.  Plans  for  sufficient  time  prior  to  maneuvers  for  programming  of  automated  systems. 

9.  Ensure  that  all  crewmembers  are  aware  of  status  and  changes  in  automated 
systems  parameters. 

1 0.  Crewmembers  recognize  potential  distractions  posed  by  automation  and  take 
appropriate  preventive  action ,  including  reversion  to  lower  levels  of  automation. 

O.  Operational  Integrity 

Mission  Completion  and  Risk  Management.  This  rating  assesses  the  ability  to  complete 

the  mission  while  minimizing  risk  and  avoiding  catastrophe. 

1 .  When  confronted  with  an  unusual  event,  evaluate  options  thoroughly  without 
becoming  committed  to  a  single  course  of  action  with  a  high  degree  of  risk. 

2.  Preformulate  strategies  to  deal  effectively  with  critical  operational  situations. 


NASA/UT  October  1990 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  A-29 


3.  Consider  desired  vs.  expected  outcome.  Project  state  or  condition  of  aircraft  to 
expected  terminal  condition  and  map  this  to  desired  condition. 

Regulatory  Compliance.  This  rating  evaluated  the  adherence  to  all  pertinent  operational 
regulations  including  FAR's,  operational  specifications,  etc. 

1.  Conscientious  use  of  SOP'S. 

2.  Intentional  non-compliance  when  this  is  judged  to  be  the  safest  course  of  action. 

Schedule  Reliability.  Significant  delays  will  impose  schedule  and  economic  penalties  and 
must  be  considered  during  the  conduct  of  the  operation. 

1.  Mission  plan  factors  in  destination  arrival  time. 

2.  Enroute  decisions  factor  in  destination  arrival  time. 

3.  Anticipated  operational  delays  are  communicated  promptly. 

Operational  Economy.  This  rating  involves  considerations  of  efficient,  economic  flight 
operations  performance.  Fuel  conservation  is  a  significant  component  of  economy. 

1 .  Enroute  burn-out  considered  as  part  of  all  major  planning  decisions. 

2.  Approach  speeds,  flap  settings  and  runway  length  are  considered  to  minimize  brake 
wear. 


II.  Crew  Performance  Markers-Technical  Factors 

This  area  focuses  on  the  crew  as  a  unit  and  how  well  they  discharge  the  technical 
aspects  of  the  mission.  It  specifically  addresses  precision  maneuvers;  propulsion,  lift 
and  drag  control;  systems  operations  and  malfunction  warning  and  reconfiguration. 

A.  Flight  Maneuvers  and  Attitude  Control. 

Flight  maneuvers  involve  precision  manipulation  of  the  velocity  vector  of  the  aircraft  in 
time  and  space.  Attitude  control  involves  the  maintenance  of  appropriate  longitudinal 
and  lateral  transition  and  terminal  positioning.  Major  focus  here  is  on  the  "performance 
and  control"  aspects  of  flying. 


NASA/UT  October  1990 


Page  A-30 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


1.  Instrument  Interpretation:  properly  interpret  control,  performance  and  AFDS 
indications. 

2.  Instrument  Selection:  select  optimum  displays  and  functions  for  specific  mission 
tasks. 

3.  Lateral  Navigation:  predict  and  execute  optimum  course  intercept,  capture  and 
track  optimum  course  to  achieve  specific  mission  objective. 

4.  Vertical  Navigation:  monitor  or  control  descent  rate  and/or  path  to  achieve  target 
vertical  points.  Maneuver  aircraft  to  maintain  optimum  descent  path  and  recognize 
status  of  the  velocity  vector. 

5.  Attitude  Control:  maintain  optimum  pitch  attitude  and  bank  angle  control  for 
appropriate  flight  conditions. 

6.  Velocity  Vector  Control:  maintain  optimum  velocity  vector  control  to  achieve  target 
vertical  points. 

7.  Trim  Control:  maintain  "in  trim"  condition  to  achieve  a  stable  platform. 

8.  Assess  Aircraft  State:  maintain  position/terr.ain/obstacle  awareness. 

9.  Approach  Stabilization:  achieve  and  maintain  a  stabilized  final  approach  state. 
Considerations  here  include: 

a)  Configuration  management 

b)  Speed  management 

c)  Glideslope  control 

d)  Power  management 

e)  Localizer  heading  control 

f)  In  close  line  up,  attitude  and  wings  control. 

B.  Propulsion/Lift/Drag  Control 

Propulsion,  lift  and  drag  control  involves  achieving  the  optimum  aircraft  energy  state  to 

in  turn  achieve  a  specific  mission  objective. 

1 .  Instrument  Interpretation:  proper  and  timely  entre  of  key  propulsion,  lift  and/or  drag 
indications. 
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2.  Energy  Management:  achieve  and  maintain  target  airspeed  consistent  with  target 
altitudes  and  mission  constraints.  Optimum  employment  of  aircrafts'  acceleration/ 
deceleration  rates. 

3.  Power  Control:  smooth,  precise  application  of  power  to  achieve  desired  mission 
objectives. 

4.  Lift  Control:  optimize  lift  control  systems  to  achieve  desired  mission  objectives 
within  known  mission  constraints. 

5.  Drag  Control:  manage  aircraft  platform  and  aerodynamic  drag  to  achieve  desired 
deceleration  rates. 

C.  System  Operations 

Systems  operations  involve  the  location  and  interpretation  of  controls  and  displays;  the 
knowledge  of  operating  concepts/limits;  and  the  demonstrated  use  of  operating 
procedures  to  accomplish  specific  mission  objectives. 

1 .  Instrument  Interpretation:  location  of  pertinent  instruments  and  the  interpretation  of 
mission  critical  information. 

2.  Operating  Concepts:  system  operating  concepts  and  functions  that  permit  timely 
utilization  of  all  system  resources. 

3.  Operating  Limits:  limitations  of  system  capabilities  that  may  impact  mission 
performance. 

4.  Operating  Procedures:  normal  operating  procedures  that  permit  timely  and  effective 
utilization  of  all  key  system  functions. 

D.  Malfunction  Warning  and  Reconfiguration 

Malfunction  warning  involves  assessing  the  information  provided  that  indicates  a 
degradation  of  system  capability  and/or  integrity.  System  reconfiguration  involves  the 
means  by  which  a  system-aircraft  is  configured  for  continued  safe  flight  after  an 
abnormality  has  occurred. 

1.  Instrument  Interpretation:  assessment  of  mission-critical  information  indication  that 
a  system/aircraft  degradation  has  occurred. 
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2.  Operating  Concepts:  system  operating  concepts  that  permit  timely  warnings  of 
actual  or  impending  system  failures. 

3.  Operating  Procedures:  system/aircraft  reconfiguration  procedures  to  ensure 
continued  safe  flight  is  possible. 

E.  Energy  Management 

Effective  Energy  Management  requires  the  use  of  altitude,  airspeed,  fuel  and  angular 
position  to  arrive  at  a  predetermined  flight  fix  (or  portion  in  space)  at  a  specified  time 
while  maintaining  precise  flight  trajectory  and  attitude  control. 

Establishing  an  optimum  flight  trajectory  profile  while  maintaining  target  state 
parameters  is  the  ultimate  aim  of  energy  management  and  requires  significant  skill  to 
perform  this  in  a  modern  low  drag,  high  performance  aircraft. 

Extensive  studies  have  been  conducted  relating  to  energy  management  for  military 
tactical  aircraft  (see  Bridenbach,  Ciavarelli,  1985).  Most  observers  have  seen  little 
application  for  this  research  in  the  civil  transport  environment.  However,  recent  interest 
in  optimal  recovery  techniques  for  windshear  encounters  (Meile,  et.  al.,  1990)  suggest 
that  more  training  emphasis  may  be  needed  in  energy/trajectory  management,  not  only 
for  unusual  windshear  encounters  but  also  for  the  complete  approach  and  landing  phase 
of  the  transport  mission. 

Energy.  For  the  purpose  of  this  report  "energy"  can  be  defined  as  consisting  of  the 
following: 

1 .  Airspeed 

2.  Altitude 

3.  Angular  position  (relative  to  some  point) 

4.  Fuel 

5.  Thrust  available 

6.  Drag 

Management.  For  the  approach  and  landing  phase  of  the  transport  mission, 
"management"  of  energy  consists  of: 

1 .  Control  of  thrust 

2.  Control  of  drag 

3.  Optimization  of  flight  trajectory 

4.  Optimization  of  angular  position 

5.  Optimization  of  fuel 
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6.  Optimization  of  descent  rate 

7.  Optimization  of  speed  decay  rate 

From  the  above  it  can  be  seen  that  "Energy  Management"  is  a  conceptually  complex 
aircrew  activity  that  clearly  needs  a  high  degree  of  skill  for  successful  execution. 


III.  SUMMARY 

The  Mission  Performance  Model,  complete  with  each  crew  performance  marker  is 
shown  in  Figure  1 .  Here,  human  factors  as  well  as  technical  performance  clusters  are 
specified  along  with  the  applicable  markers  under  each  cluster.  For  example,  under 
workload  management  and  situational  awareness,  key  markers  include:  preparation, 
planning,  vigilance,  workload  distribution,  and  distraction  avoidance. 

Similarly,  under  the  cluster  entitled  "Propulsion/Lift/Drag  Control";  key  markers  include: 
instrument  interpretation,  energy  management,  power  control,  lift  control  and  drag 
control.  When  all  these  markers  are  combined  into  their  various  categories,  the  concept 
of  a  mission  performance  model  emerges. 
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I.  SCHEDULE 


Dav  I 


•  Introductions,  Logistics,  Administrative. 

•  Presentations  on  CRM  and  the  Mission  Performance  Model: 

>  Capt.  Kevin  Smith  and  Capt.  Bill  Hamman,  United  Air  Lines 

>  Mr.  Jan  Demuth,  FAA 

•  BREAK 

•  Distribution  of  Workshop  materials/protocols;  Discussion  of  Phase  I  materials  and 
Phase  II  (Workshop)  objectives  and  procedures. 

•  Group  Assignments 

•  LUNCH 

•  Group  Processes 

•  Begin  Objectives 

Dav  IT  and  III 

•  Working  the  Objectives  -  at  some  point,  finalize;  IOU’s;  discuss  next  step(s). 


Page  B-2 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


II.  WORKING  OBJECTIVES 

A.  Objective  I: 

1.  To  examine  the  performance  markers-both  CRM  and  Technical. 

2.  To  examine  the  two  behaviorally  anchored  rating  scale(s)  for  the  above—  both  of  these 
'can  be  (initially  and  individually)  a  "homework"  pan  of  Phase  I;  the  finalization  via 
group  work  will  do  at  the  workshop. 

B.  Objective  II:  To  complete  the  mission  performance  model  by: 

1.  Developing  a  criticality  scale/rating  methodology  (or,  other  method)  for  the 
identification  and  criticality-rating  of  the  sub-tasks/elements  on  any  mission  task. 

ALTERNATE:  To  compare/contrast/modify/integrate  (—whatever)  the  MPM  and  Figure  2  in 
the  latest  draft  of  the  CRM  Advisory.  (Remainder  of  objective  is  somewhat  the  same.) 

2.  Exercise  the  model-use  3  tasks  per  phase  of  flight  (normal;  abnormal/  emergency; 
"other")  as  an  initial  review/critique  and  modification;  as  a  check  on  the  completeness  of 
the  task  decomposition  via  criticality-rating  (or,  whatever  other  method  was  used). 

3.  Re-exercise  the  model  to  check  any  mods  that  were  made  (—same  format  as  in 
II.B,  IV.. 5  as  to  the  number  of  tasks,  etc.). 

4.  Identify/articulate  any  rules  and  algorithms  embedded  in  the  model. 

5.  Develop  scales/rules  for  evaluating  the  existence/strength  of  CRM  and  technical  skill 
interrelationships  —  task/sub-task  level. 

C.  Objective  HI;  To  identify/examine  the  potential  uses  of  the  model-with  attention  to: 

1.  Aircrew  CRM  Training  development; 

2.  LOFT  scenario  development  and/or  evaluation; 

3.  Instructor  Pilot/Evaluator  training; 

4.  CRM/LOFT  evaluation. 

—The  latest  draft  of  the  CRM  AC  will  be  used  here  and  in  Objective  II  as  key  references/input. 
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III.  GROUP  PROCESS 

A.  Group  Functioning  [omitted  from  final  report] 

B.  Group  Decision-making 

There  are  a  number  of  approaches  which  groups  can  use  to  reach  decisions  at  various  points 
in  their  problem  solving  activities.  These  are  pros  and  cons  to  all  these  approaches. 
However,  the  approach  which  has  the  most  potential  for  participation  and  interaction  is 
consensus. 

Consensus  -  occurs  when  a  final  solution  is  reached  which  is  acceptable  to  everyone  (Filley, 
1975).  Consensus  must  be  achieved  through  discussion  in  such  a  way  that: 

1 .  no  group  member(s)  is  (are)  placed  in  a  win-lose  position; 

2.  the  group  members  do  not  waste  time  in  only  defending/supporting  their 
respective  positions— to  the  exclusion  of  real  discussion;  and, 

3.  group  cohesion  is  not  disrupted. 

Consensus  may  lead  to  "regression  toward  the  mean;"  that  is,  in  order  to  maintain  group 
cohesion,  individuals  may  distort  their  independent  judgements  to  accommodate  each  other. 
This  is  to  be  watched  for  and  avoided.  Finally,  while  consensus  can  be  a  very  powerful 
approach  to  decision  making,  it  may  fall  far  short  of  its  potential  if  the  group  "chemistry"  is 
improper  and/or  the  group  facilitator/leader  does  not  have  the  necessary  skills. 


C.  Some  Guidelines  for  Effective  Consensus 

(The  guidelines  are  research-based  and  can  be  used  as  general  directions  in  practically  any 
group  problem-solving  activity.  Groups  that  actively  attempt  to  follow  these  guidelines  will 
get  quality  results.) 

1 .  Always  strive  for  the  best  answer,  given  the  data  you  have  to  work  with. 
Excellence  is  always  desirable  and  should  be  actively  pursued. 

2.  Assume  that  the  task  can  be  done  and  that  some  solutions  may  be  better  than 
others.  Do  not  attach  your  self-worth  to  the  selection  of  a  particular  item.  Do 
not  assume  that  someone  must  win  and  someone  must  lose  when  discussion 
reaches  a  stalemate.  Try  not  to  compete.  Even  if  you  win,  the  group  may 
lose. 
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3.  Present  your  position,  but  listen  to  other  group  members’  reactions  and  consider 
them  carefully  before  you  press  a  particular  point.  State  your  own  beliefs  or 
opinions  concerning  the  issues,  but  make  certain  that  all  group  members  are 
heard  from  and  contribute  to  the  discussion. 

4.  Attempt  to  involve  everyone  in  the  discussion  and  decision  making  process. 
Disagreements  can  enhance  the  quality  of  the  decision  because  a  wide  range  of 
information  and  opinion  maximizes  the  chance  that  a  group  will  make  a  more 
adequate  choice.  Treat  differences  of  opinion  as  a  means  of  gathering  more 
information,  clarifying  issues,  and  causing  the  group  to  seek  better  alternatives. 


D.  Groupthink  Versus  Consensus 

The  basic  differences  are  simple  in  theory,  but  difficult  to  overcome  in  practice.  From  the 
outside  of  a  group,  watching  it  as  it  were,  consensus  and  groupthink  can  seem  to  be  much 
alike,  and  groupthink  can  be  so  seductive  and  feel  so  good  that  it  is  easy  to  fall  into  and, 
even  easier,  to  believe  that  it  is  consensus. 

Consensus  involves  work;  often  it  doesn’t  feel  "good"  as  you  work  towards  it,  but  it  holds 
together  later  and  the  group  is  more  cohesive  and  better  functioning  as  a  result. 

Groupthink,  which  is  the  single  biggest  barrier  to  real  consensus,  seems  to  feel  good  as  it 
happens.  It  builds  nothing,  however,  except  the  illusion  of  harmony  and  the  feelings  (in 
some/all  members)  that  something  isn’t  quite  right.  But  the  rest  of  the  group  seems  happy 
and  the  task  seems  to  flow.  Members  self-censor  to  keep  up  this  illusory  harmony.  There 
is  no  real  critical  thought  and  the  "harmony"  can  disappear  in  a  flash. 


E.  Groupthink  Symptoms 

Groupthink  encompasses  a  number  of  symptoms;  groupthink  exists  in  a  group  to  the  degree 
that  these  eight  symptoms  are  present. 

1 .  Illusion  of  unanimity  regarding  the  viewpoint  held  by  the  majority  in  the  group 
and  an  emphasis  on  team  play. 
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2.  A  view  of  the  "opposition"  as  generally  inept,  incompetent,  and  incapable  of 
countering  effectively  any  action  by  the  group,  no  matter  how  risky  the  decision 
or  how  high  the  odds  are  against  the  plan  of  action  succeeding. 

3.  Self-censorship  of  group  members  in  which  overt  disagreements  are  avoided; 
facts  that  might  reduce  support  for  the  emerging  majority  view  are  suppressed; 
faulty  assumptions  are  not  questioned,  and  personal  doubts  are  suppressed  in  the 
form  of  group  harmony. 

4.  Collective  rationalization  to  comfort  one  another  in  order  to  discount  warnings 
that  what  the  group  seems  to  have  agreed  upon  is  either  unworkable  or  highly 
unlikely  to  succeed. 

5.  Self-appointed  mindguards  within  the  group  that  function  to  prevent  anyone 
from  undermining  its  apparent  unanimity  and  to  protect  its  members  from 
unwelcome  ideas  and  adverse  information  that  may  threaten  consensus. 

6.  Reinforcement  of  any  seeming  consensus  by  direct  pressure  on  any  dissenting 
group  member  who  expresses  strong  reservations  or  challenges,  or  argues 
against  the  apparent  unanimity  of  the  group. 

7.  A  shared  feeling  of  unassailability  marked  by  a  high  degree  of  esprit  de  corps, 
by  implicit  faith  in  the  wisdom  of  the  group,  and  by  an  inordinate  optimism  that 
disposes  members  to  take  excessive  risks. 


IV.  PROTOCOLS 


Phase  II  CRM  Integration  Workshop 

NOTE:  A  7-page  set  of  protocols  was  provided  to  the  Workshop  participants.  Due  to 

space  limitations  and  to  the  various,  on-the-spot  changes  made  during  the 
Workshop  itself,  these  are  not  reproduced  here.  If  you  are  interested  in  them, 
please  call  Dr.  Ron  Lofaro,  FAA,  at  (212)  267-8529. 
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CREW  RESOURCE  MANAGEMENT  TRAINING 
ADVISORY  CIRCULAR 
DRAFT  3.4  February  27,  1992 

CONTF.NTS 


1)  Purpose 

2)  Related  FAR  Sections  and  Advisory  Circulars 

3)  Related  Reading  Material 

4)  Definitions 

5)  Background 

6)  Basic  Concepts  of  Crew  Resource  Management 

7)  Fundamentals  of  CRM  Implementation 

8)  Components  of  CRM  Training 

9)  Suggested  Curriculum  Topics 

10)  Evolving  Concepts  of  CRM 

1 1)  Assessment  of  CRM  Training  Programs 

12)  The  Critical  Role  of  Check  Airmen  and  Instructors 
Appendix  l :  Crew  Performance  Markers 
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Crew  Resource  Management  Training  AC  No:  120-5 la 
Draft  3.4  Rev.  2/27/92 

ATA  AQP  Committee:  Advisory  Circular  Focus  Group  1/7/92 
Air  Carrier  Training  Working  Group  2/26/92 

1  -  PURPOSE.  This  advisory  circular  (AC)  presents  guidelines  for  developing, 
implementing,  reinforcing  and  assessing  training  programs  in  the  human  factors  of  flight 
operations  for  flight  crewmembers  and  other  personnel.  These  programs  are  known  as 
Crew  Resource  Management  and  are  designed  to  become  an  integral  part  of  training  and 
operations. 

2.  RELATED  FAR  SECTIONS  and  ADVISORY  CIRCULARS. 

a.  Part  121.  Subnart  N  (Training).  121.400-405,  121.409-422,  121.424, 
121.427. 

b.  Part  121.  Suboart  0  (Crewmember  Qualifications).  121.432-433,  121.434, 
121.440-443. 

c.  Part  135.  Suboart  E  (Flight  Crewmember  Requirements.  135.243-245. 

d.  Part  135.  Subpart  G  (Crewmember  Testing  Requirements).  135.293-295, 
135.299-301. 

e.  Part  135.  Subpart  H  (Training).  135.321-331.  135.335-351. 

f.  Advisory  Circular  120-35B  Line  Oriented  Simulations 

g.  SFAR  58.  Advanced  Qualification  Program 
Advanced  Qualification  Program  Advisory  Circular 

3.  RELATED  READING  MATERIAL.  For  detailed  information  on  the  recommendations 
made  in  this  AC,  the  reader  is  encouraged  to  review  Crew  Resource  Management:  An 
Introductory  Handbook  published  by  the  FAA  (tentative  title  and  publication).  Additional 
background  material  can  be  found  in  Cockpit  Resource  Management  Training: 
Proceedings  of  a  NASA/MAC,  Workshop ,  1987.  The  National  Aeronautics  and  Space 
Administration  (NASA)  Conference  Proceedings  (CP)  number  is  2455.  The  National  Plan 
for  Aviation  Human  Factors  defines  research  issues  related  to  crew  coordination  and 
training.  Copies  may  be  purchased  from  the  National  Technical  Information  Service, 
U.S.  Department  of  Commerce,  5285  Port  Royal  Road,  Springfield,  Virginia  22161, 
(703)  487-4650.  Descriptions  of  current  research  and  organizational  experience  can 
be  found  in  Helmreich  &  Wilhelm  (1992)  "Outcomes  of  CRM  training",  International 
Journal  of  Aviation  Psychology  and  in  E.L.  Wiener,  B.G.  Kanki,  &  R.L.  Helmreich  (Eds.). 
(In  press).  Cockpit  Resource  Management.  Orlando:  Academic  Press. 

4.  DEFINITIONS. 

a.  Human  Factors.  Human  factors  is  a  multidisciplinary  field  that  draws  on  the 
methods  and  principles  of  the  behavioral  and  social  sciences,  engineering,  and 
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physiology  to  optimize  human  performance  and  reduce  human  error.  In  short,  human 
factors  has  become  an  applied  science  of  people  working  together  with  machines.  Just 
as  individual  errors  can  degrade  the  performance  and  safety  of  a  system  because  of  the 
way  the  hardware  is  designed  or  because  of  inadequate  operator  training,  so  too  can 
errors  in  the  design  and  management  of  crew  tasks  and  of  organizations  degrade 
system  performance. 

b.  Crew  Resource  Management.  The  application  of  human  factors  concepts  in 
the  flightdeck  environment  was  initially  known  as  Cockpit  Resource  Management.  More 
recently,  as  human  factors  programs  have  come  to  include  other  participants  in  the 
aviation  system  such  as  cabin  crews  and  maintenance  personnel,  the  phrase  Crew 
Resource  Management  (CRM)  has  come  into  wide  use.  CRM  refers  to  the  effective  u§e 
of  all  available  resources,  human,  hardware,  and  informational.  A  more  modern 
definition  includes  not  only  the  cockpit  crew  but  also  all  other  groups  that  routinely 
work  with  that  crew  and  are  involved  in  decisions  required  to  operate  a  flight  safely. 
These  groups  include  but  are  not  limited  to: 

(1)  dispatchers 

(2)  cabin  crewmembers 

(3)  maintenance  personnel 

(4)  air  traffic  controllers 

CRM  encompasses  optimizing  both  the  person-machine  interface  and  interpersonal 
activities  including  effective  team  formation  and  maintenance,  information  transfer, 
problem  solving,  decision  making,  maintaining  situation  awareness,  and  dealing  with 
automated  systems.  Training  in  CRM  thus  involves  initial  indoctrination  and  recurrent 
training  and  reinforcement  of  crews  in  human  factors  concepts  in  the  aviation  system. 

5.  BACKGROUND. 

a.  Investigations  into  the  causes  of  air  carrier  accidents  have  shown  that  human 
error  is  a  contributing  factor  in  between  60  and  80  percent  of  all  air  carrier  incidents  and 
accidents.  This  AC  provides  guidelines  for  FAR  Parts  121  and  135  certificate  holders 
to  establish  human  factors  p  '•grams  designed  to  increase  the  efficiency  with  which 
flight  crewmembers  interact  in  the  cockpit  by  focusing  on  communication  skills, 
teamwork,  task  allocation,  and  decision  making. 

b.  A  long-term  NASA  research  program  has  demonstrated  that  these  types  of 
incidents  have  many  common  characteristics.  One  of  the  most  compelling  observations 
of  this  program  and  other  research  studies  is  that  many  problems  encountered  by 
flightcrews  have  very  little  to  do  with  the  more  technical  aspects  of  operating  a 
multi-person  crew  aircraft.  Instead,  they  are  associated  with  poor  group  decision 
making,  ineffective  communication,  inadequate  leadership,  and  poor  management. 
Training  programs  historically  emphasized  almost  exciusively  the  technical  aspects  of 
flying  and  do  not  deal  effectively  with  various  types  of  crew  management  strategies  and 
techniques  that  are  also  essential  to  safe  flight  operations. 

c.  These  observations  have  recently  led  to  a  developing  consensus  in  both 
industry  and  government  that  more  training  emphasis  needs  to  be  placed  upon  the 
factors  that  influence  crew  coordination  and  the  management  of  crew  resources.  CRM 
training  programs  have  been  or  are  being  developed  by  most  major  and  some  regional 
air  carriers. 
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d.  A  1 986  NASA  workshop  on  CRM  training  attended  by  representatives  of  many 
segments  of  the  aviation  community,  has  produced  a  series  of  recommendations  for 
training  programs  in  this  area.  These  guidelines,  while  not  mandatory,  provide  useful 
background  information  for  understanding  the  critical  elements  of  human  factors 
training. 


e.  Continuing  research  sponsored  by  NASA  and  the  FAA  evaluating  the  impact 
of  CRM  training  indicates  that  initial  indoctrination  causes  significant  improvement  in 
attitudes  regarding  crew  coordination  and  flightdeck  management.  In  programs  that 
also  provide  recurrent  training  and  opportunities  to  practice  CRM  concepts,  significant 
changes  in  observed  behavior  on  the  flightdeck  and  crew  performance  during  Line 
Oriented  Flight  Training  have  been  found.  Crews  with  CRM  training  operate  more 
effectively  as  teams  and  cope  more  effectively  with  non-standard  situations. 

f.  This  research  also  shows  that  when  there  is  not  effective,  recurrent  training 
in  and  reinforcement  of  CRM  concepts,  improvements  in  attitudes  found  after  initial 
indoctrination  tend  to  disappear  and  revert  to  former  levels. 

6.  BASIC  CONCEPTS  OF  CREW  RESOURCE  MANAGEMENT. 

While  there  are  many  approaches  and  techniques  useful  in  CRM  training,  it  seems 
clear  that  certain  features  are  necessary.  The  training  should  focus  on  the  functioning 
of  crews  as  intact  teams ,  not  simply  as  a  collection  of  technically  competent  individuals. 
The  training  should  provide  opportunities  for  crewmembers  to  practice  the  skills  that  are 
necessary  to  be  good  team  leaders  and  team  members.  This  requires  that  training 
exercises  include  all  crewmembers  functioning  in  the  same  roles  they  normally  perform 
in  flight.  CRM  training  should  help  crewmembers  learn  how  to  behave  in  ways  that 
foster  crew  effectiveness.  It  should  also  help  crewmembers  learn  that  how  they  behave 
during  normal,  routine  circumstances  can  have  a  powerful  impact  on  how  well  they 
function  during  high  workload,  stressful  situations.  During  emergency  situations,  it  is 
highly  unlikely  (and  probably  undesirable)  that  any  crewmember  will  take  the  time  to 
reflect  upon  his  or  her  CRM  training  to  figure  out  how  to  act.  However,  actions 
normally  practiced  during  more  relaxed  times  increase  the  chances  that  a  crew  will 
handle  stressful  situations  more  competently. 

Findings  from  research  show  that  lasting  behavior  change  in  any  environment 
cannot  be  achieved  in  a  short  period,  even  if  the  training  is  very  well  designed.  Trainees 
need  time,  awareness,  practice  and  feedback,  and  continuing  reinforcement  to  learn 
lessons  that  will  endure  over  long  periods  of  time.  In  order  to  be  effective,  CRM 
concepts  must  be  integrated  into  all  aspects  of  training  and  operations. 

CRM,  is  defined  by  the  following  basic  characteristics: 

(a)  It  is  a  comprehensive  system  of  applying  human  factors  concepts  to  improve  crew 
performance 

(b)  It  is  designed  for  all  operational  personnel. 

(c)  It  can  be  extended  to  all  forms  of  aircrew  training. 

(d)  It  concentrates  on  crewmember  attitudes  and  behaviors  and  their  impact  on  safety. 

(e)  It  uses  the  crew  as  the  unit  of  training. 
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(f)  It  is  training  that  requires  the  active  participation  of  all  crewmembers.  It  provides 

an  opportunity  for  individuals  and  crews  to  examine  their  own  behavior  and  make 

decisions  on  how  to  improve  cockpit  teamwork. 

(g)  It  uses  check  airmen  and  instructors  who  are  highly  qualified  and  specially  trained 

in  CRM. 

(i)  Line  Oriented  Flight  Training  (LOFT)  sessions  provide  an  extremely  effective 
means  of  practicing  CRM  skills  and  receiving  feedback  and  reinforcement. 

(ii)  Video  feedback  during  debriefing  of  LOFT  and  other  training  should  optimally 
be  provided  so  that  crewmembers  could  asses  their  skills  as  individuals  and 
team  members. 

(iii)  In  cases  where  simulators  are  not  available,  crewmembers  can  participate  in 
group  problem-solving  exercises  designed  to  exercise  CRM  skills.  Through 
video  feedback  during  debriefing,  they  can  then  assess  the  positive  and 
negative  actions  of  all  crewmembers. 

(iv)  Crewmembers  may  also  participate  in  role-playing  exercises  designed  to 
provide  practice  in  developing  strategies  for  dealing  with  incidents  and  to  allow 
analyses  of  behaviors  during  incidents.  Again,  video  feedback  is  useful  for 
assessment  and  feedback  during  debriefing  of  crew  abilities  in  such  areas  as 
decision  making,  team  participation,  and  leadership. 

(iv)  Attitude  and/or  personality  measures  can  also  be  used  to  provide  feedback  to 
participants,  thereby  allowing  them  to  assess  their  individual  strengths  and 
weaknesses. 

7.  FUNDAMENTALS  OF  CRM  IMPLEMENTATION 

Both  research  and  the  operational  experience  of  airlines  implementing  CRM 
programs  have  suggested  that  greatest  impact  is  achieved  if  a  number  of  steps  are 
taken.  The  following  are  actions  that  have  been  observed  to  increase  program 
effectiveness.  A  number  of  these  issues  are  discussed  in  more  detail  in  later  sections. 

a.  Demonstrate  total  commitment  to  the  program.  Programs  are  received  much 
more  positively  when  senior  management,  flight  operations,  and  flight  standards 
personnel  publicly  support  the  concepts  and  provide  necessary  resources.  In  addition 
Flight  Operations  and  Training  Manuals  should  embrace  CRM  concepts  and  provide 
crews  with  necessary  procedures,  policy  guidance,  and  training. 

b.  C^.  nmunicate  the  nature  and  scope  of  the  program  prior  to  startup.  Providing 
crews  with  an  expectation  of  what  the  training  will  involve  and  plans  for  initial  and 
continuing  training  can  prevent  misunderstandings  regarding  the  focus  of  the  training 
and  its  implementation. 

c.  Assess  the  status  of  organizations  before  implementation.  It  is  important  to 
know  how  widely  CRM  concepts  are  understood  and  practiced  before  designing  specific 
training.  Surveys  of  flightcrews,  observation  of  crews  in  line  observations,  and  analysis 
of  operational  incident  reports  can  all  provide  essential  data  for  program  designers. 
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d.  Customize  the  training  to  reflect  the  nature  and  needs  of  the  organization. 
Using  knowledge  of  the  state  of  the  organization,  priorities  should  be  established  for 
topics  to  be  covered  including  special  issues  such  as  the  effects  of  mergers  or  the 
introduction  of  advanced  technology  aircraft.  This  approach  can  increase  the  relevance 
of  training  for  crewmembers. 

e.  Define  the  scope  of  the  program.  Institute  special  CRM  training  for  key 
personnel  including  check  airmen  and  instructors.  It  is  useful  to  provide  training  for 
these  groups  prior  to  beginning  training  for  line  crews.  CRM  training  may  be  expanded 
to  include  dispatchers,  cabin  crews,  maintenance  personnel  and  other  team  members 
as  appropriate.  It  is  also  helpful  to  develop  a  long  term  strategy  for  program 
implementation. 

f.  Institute  quality  control  procedures.  It  has  proved  helpful  to  monitor  the 
delivery  of  training  and  to  determine  areas  where  training  can  be  strengthened.  This 
can  be  accomplished  by  providing  special  training  for  program  instructors  (usually  called 
facilitators)  and  using  surveys  to  collect  systematic  feedback  from  participants  in  the 
training. 

8.  COMPONENTS  OF  CRM  TRAINING. 

Overall  objectives  of  CRM.  CRM  training  is  designed  to  prevent  incidents  and 
accidents  and  to  improve  crew  coordination  and  performance.  The  following 
components  have  been  identified  as  critical  for  effective  CRM  implementation.  They  do 
not  represent  a  fixed  sequence  of  presentation.  In  practice,  many  airlines  combine 
elements  of  each  in  both  initial  and  recurrent  training  and  in  checking. 

a.  Initial  Indoctrination/Awareness. 

(1)  Indoctrination/awareness  typically  consists  of  classroom  presentations  and 
focuses  on  communications  and  decision  making,  interpersonal  relations  and  crew 
coordination,  and  leadership.  In  this  component  of  CRM  programs  the  concepts  are 
developed,  defined,  and  related  to  line  operations  and  the  safety  of  operations.  This 
part  of  the  training  also  provides  a  common  terminology  and  conceptual  framework  for 
identifying  and  describing  crew  coordination  problems. 

(2)  This  can  be  accomplished  by  a  combination  of  training  methods  such  as  lecture 
presentations,  discussion  groups,  role-playing  exercises,  computer-based  instruction, 
and  videotape  examples  of  good  and  poor  team  behavior.. 

(3)  Initiating  indoctrination/awareness  training  includes  the  development  of  a 
curriculum  that  addresses  CRM  skills  that  have  been  demonstrated  to  influence  crew 
performance.  To  be  most  effective,  the  curriculum  should  both  define  the  concepts 
involved  and  relate  directly  to  operational  issues  crews  face.  Many  organizations  have 
found  it  useful  to  survey  crewmembers  to  determine  attitudes  regarding  crew 
coordination  and  cockpit  management  and  to  examine  operational  problems  to  prioritize 
training  issues  to  address  topics  of  greatest  operational  significance. 

(4)  Effective  indoctrination/awareness  training  increases  the  credibility  of  the 
concepts  and  helps  in  changing  attitudes  to  be  more  favorable  to  human  factors 
concerns.  It  may  also  demonstrate  more  effective  communications  practices  and 
enhance  interpersonal  skills.  However,  it  is  important  to  realize  that  this  is  only  a 
necessary  first  step.  Many  programs  rely  almost  exclusively  on  this  aspect  of  training. 
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but  classroom  instruction  alone  will  not  fundamentally  alter  crewmember  attitudes  and 
behavior  over  the  long  term. 

b.  Recurrent-Practice  and  Feedback. 

(1)  CRM  training  should  be  instituted  as  a  regular  part  of  the  recurrent  training 
requirement.  Recurrent  CRM  training  should  include  refresher  curriculum  and  practice 
and  feedback  exercises  such  as  LOFT  with  video  feedback,  or  a  suitable  substitute  such 
as  a  flight  training  device  employing  video  feedback.  It  is  particularly  important  that 
some  of  these  recurrent  CRM  exercises  take  place  with  a  full  crew-each  member 
operating  in  their  normal  crew  position.  For  example,  recurrent  training  LOFT  exercises 
designed  for  CRM  should  be  conducted  with  actual  crews  as  much  as  possible. 

(2)  Recurrent  practice  and  feedback  training  allows  participants  to  employ  newly 
acquired  skills  in  communications  and  interpersonal  relationships  and  to  receive 
feedback  on  their  effectiveness.  Feedback  has  great  impact  when  it  comes  from  self¬ 
critique  and  peers  with  the  guidance  of  a  facilitator  with  special  training  in  assessment 
and  debriefing  techniques.  Effective  feedback  will  reflect  the  operational  factors  defined 
in  the  Indoctrination/awareness  Phase  and  will  relate  to  specific  behaviors.  Practice  and 
feedback  are  best  accomplished  through  the  use  of  simulators  or  training  devices  and 
video  recording.  Video  feedback,  under  the  guidance  of  a  facilitator  is  particularly 
effective  because  it  allows  participants  to  view  themselves  from  a  third-person 
perspective;  this  promotes  acceptance  of  one's  weak  areas,  which  encourages  attitude 
and  behavioral  change. 

c.  Continuing  Reinforcement 

(1)  No  matter  how  effective  the  classroom  curriculum,  interpersonal  drills,  LOFT 
exercises,  and  feedback  techniques  are,  a  single  exposure  will  be  insufficient.  The 
attitudes  and  norms  that  contribute  to  ineffective  crew  coordination  have  developed 
over  a  crewmember's  lifetime.  It  is  unrealistic  to  expect  a  short  training  program  to 
reverse  years  of  habits.  To  be  maximally  effective,  CRM  should  be  embedded  in 
training  and  human  factors  concepts  should  be  stressed  in  line  operations  and 
evaluation. 

(2)  CRM  should  become  an  inseparable  part  of  the  organization's  culture. 

(3)  There  is  a  natural  tendency  to  think  of  CRM  as  training  only  for  the  "managers" 
or  captains.  However,  this  notion  misses  the  essence  of  the  primary  CRM  training 
objective-the  prevention  of  crew-related  incidents  and  accidents.  It  should  be  most 
effective  in  the  entire  crew  context,  and  this  requires  training  exercises  that  include  all 
crewmembers  working  together  and  learning  together.  In  the  past,  much  of  flightcrew 
training  has  been  separated  by  crew  position,  and  while  this  may  be  effective  for  certain 
types  of  training  (e.g.,  technical  skills  and  systems  knowledge,  specialized  issues  in 
upgrade  training,  etc.),  it  should  not  be  exclusively  employed  in  CRM  training. 

(4)  Reinforcement  can  be  accomplished  in  many  different  areas.  Training  such  as 
joint  cabin  and  cockpit  crew  training  in  security  can  deal  with  many  human  factors 
areas.  Dispatch,  maintenance,  and  gate  agent  training  can  also  include  and  reinforce 
CRM  concepts. 
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The  following  topics  have  been  included  in  many  current  CRM  programs  and  have 
proved  useful.  However,  the  specific  content  of  training  and  organization  of  topics 
should  reflect  organizational  cultures  and  needs.  Appendix  1  provides  an  example.set 
of  behavioral  markers  for  each  topic.  These  markers  may  be  helpful  in  curriculum 
development  and  in  LOFT  design. 

(i)  Communications  Processes  and  Decision  Behavior.  This  area  includes 
internal  and  external  influences  on  interpersonal  communications.  External  factors 
include  barriers  such  as  rank  age,  and  gender  and  organizational  cultures.  Internal 
factors  include  listening  and  decision  making  skills,  conflict  resolution  techniques, 
and  the  use  of  appropriate  assertiveness  and  advocacy.  Specific  topics  could 
include  the  following: 

(a)  Briefings  (conduct  and  Quality).  Training  in  addressing  both 
operational  and  interpersonal  issues  and  establishing  open  communications. 

(b)  Inquirv/Advocacv/Assertion.  Showing  the  benefits  when 
crewmembers  advocate  the  course  of  action  they  feel  best  even  though  it  may 
involve  conflict  and  disagreement  with  others. 

(c)  Crew  self-critique  (decisions  and  actions).  Learning  the  uses  of 
operational  review,  feedback,  and  critique  of  actions  including  the  process  and 
the  people  involved.  To  assure  that  decisions  and  actions  taken  are 
appropriate,  it  is  essential  that  crews  review  them  during  the  course  of  flights 
with  inputs  from  all  participants.  In  addition,  one  of  the  best  techniques  to 
reinforce  effective  human  factors  practices  is  through  a  careful  debriefing  of 
activities  highlighting  the  processes  that  were  followed.  To  make  self-critique 
effective,  it  is  essential  that  each  crew  member  be  able  to  recognize  and 
assess  effective  and  ineffective  team  behavior  and  information  exchange. 

(d)  Conflict  resolution.  Defining  and  showing  means  to  resolve 
disagreements  among  crew  members  over  appropriate  courses  of  actions  or 
conflicting  information.  Demonstrating  techniques  for  maintaining  open 
communication  in  the  face  of  conflict. 

(e)  Communications  and  Decision  Making.  Demonstrating  effective 
techniques  of  seeking  and  evaluating  information.  Showing  the  influence  of 
cognitive  factors  and  biases  on  decision  quality.  Research  into  decision 
processes  in  operational  settings  and  under  stressful  conditions  is  still  at  a 
relatively  basic  level.  Nevertheless,  there  are  benefits  in  providing  crews  with 
operational  models  of  group  decision  process  that  they  can  utilize  in  situations 
where  complex  information  must  be  processed  to  make  optimal  choices. 

(ii)  Team  Building  and  Maintenance.  This  area  includes  interpersonal 
relationships  and  practices.  Effective  leadership  and  followership  along  with 
interpersonal  relationships  are  key  concepts  to  be  stressed.  Curricula  can  also 
include  recognizing  and  dealing  with  different  personalities  and  personal  operating 
styles.  Operational  factors  include: 
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(a)  Leadershio/Followership/Concern  for  task.  Showing  the  benefits  of 
the  practice  of  effective  leadership  through  coordinating  activities  and 
maintaining  proper  balance  between  authority  and  assertiveness.  Keeping  the 
goals  of  safe  and  efficient  operations  central. 

(b)  Interpersonal  relationships/Group  climate.  Demonstrating  the 
usefulness  of  showing  sensitivity  and  to  other  crewmembers'  personalities  and 
styles.  The  importance  of  recognizing  symptoms  of  fatigue  and  stress  and 
taking  appropriate  action.  Emphasizing  the  value  of  maintaining  a  friendly, 
relaxed,  and  supportive  tone  in  cockpit. 

(iii)  Workload  Management  and  Situation  Awareness.  This  topic  area 
stresses  the  importance  of  maintaining  awareness  of  the  operational  environment 
and  anticipating  contingencies  and  actions  that  may  be  required.  Instruction  may 
address  practices  (i.e.,  vigilance,  effective  planning  and  time  management,  task 
prioritizing,  avoidance  of  distractions)  that  result  in  higher  levels  of  situation 
awareness.  The  following  operational  factors  can  be  included: 

(a)  Preparation/Plannina/Vigilance.  Issues  include  devoting  appropriate 
attention  to  required  tasks,  responding  to  new  information,  and  preparing  in 
advance  for  required  activities. 

(b)  Workload  distribution/Distraction  avoidance.  Issues  involve  proper 
allocation  of  tasks  to  individuals,  avoidance  of  work  overloads,  prioritization  of 
tasks  during  periods  of  high  workload,  and  preventing  nonessential  factors 
from  distracting  attention  from  critical  tasks. 

(iv)  Individual  Factors.  Training  in  this  area  can  include  defining  and 
demonstrating  individual  characteristics  that  can  influence  crew  effectiveness. 
Research  has  shown  that  many  flightcrew  members  are  unfamiliar  with  the  negative 
effects  of  stress  and  fatigue  on  individual  and  team  performance  and  cognitive 
functions.  Training  may  include  a  review  of  scientific  evidence  on  fatigue  and 
stress  effects  including  emergencies,  personal  and  interpersonal  problems  and  the 
increased  importance  of  effective  interpersonal  communications  under  stressful 
conditions.  It  may  also  include  familiarization  with  various  countermeasures  for 
coping  with  stressors.  Additional  curriculum  topics  could  include  examination  of 
personality  and  motivational  characteristics,  self-assessment  of  personal  styles,  and 
cognitive  factors  influencing  perception  and  decision  processes. 

10.  Evolving  Concepts  of  CRM. 

a.  Extending  Training  bevond  the  cockpit.  Several  air  carriers  have  found  it  useful  to 
extend  CRM  training  both  jointly  and  with  cockpit  crews  to  other  operational  groups. 
The  goal  of  such  training  is  to  improve  the  effectiveness  of  these  groups  themselves 
and  the  quality  of  interactions  between  them  and  cockpit  crews. 

(1)  Dispatchers  share  responsibility  for  the  conduct  of  flight  with  captains  under 
the  FARS.  Both  must  agree  that  the  flight  can  be  completed  safely.  Therefore,  the 
need  to  understand  the  requirement  for  this  communication  and  what  to  expect  from 
it  is  the  underlying  reason  for  improved  interactive  training  of  pilots  and  dispatchers. 
The  basic  concepts  of  CRM  provide  a  system  to  improve  communication  skills  and 
awareness.  The  curriculum  content  and  approaches  of  initial  indoctrination,  continuing 
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practice  and  feedback,  and  continuing  reinforcement  are  relevant  to  dispatch  operations. 

(2)  Cabin  crews  form  part  of  the  team  responsible  for  every  flight.  The  extension 
of  CRM  training  to  cabin  crewmembers  provides  definition  of  team  activities  and  skills 
needed  to  make  interactions  optimum  in  both  normal  and  emergency  situations. 

(3)  Maintenance  activities  are  also  team  efforts  that  include  interactions  with  the 
cockpit.  CRM  training  for  this  group  can  enhance  both  the  maintenance  team  itself  and 
joint  activities  with  flight  crews. 

b.  Specialized  Training  in  CRM  Concepts.  As  CRM  programs  have  matured,  some 
organizations  have  found  it  advisable  to  develop  and  implement  additional  courses 
dealing  with  specific  issues  relevant  to  their  operations. 

(1)  After  all  current  crewmembers  have  completed  the  Initial 
Indoctrination/Awareness  component  of  CRM  training,  provisions  are  needed  to  provide 
newly  hired  crewmembers  with  this  background.  A  number  of  organizations  have 
modified  their  initial  course  for  presentation  to  new  crewmembers  as  part  of  initial 
training  and  qualification. 

(2)  Training  for  upgrading  to  Captain  also  provides  an  opportunity  for  specialized 
training  that  deals  with  the  human  factors  aspects  of  command.  Such  training  can  be 
incorporated  in  the  upgrade  process. 

(3)  A  number  of  human  factors  issues  related  to  the  operation  of  advanced 
technology,  highly  automated  aircraft  have  been  identified  by  NASA  and  FAA  sponsored 
research.  Issues  involving  communications  and  the  use  of  automation  can  be  developed 
for  crews  operating  these  aircraft  or  transitioning  into  them. 

11.  ASSESSMENT  OF  CRM  TRAINING  PROGRAMS. 

a.  CRM  training  is  a  dynamic  concept  that  will  continue  to  be  refined  and 
improved.  For  this  reason,  it  is  vitally  important  that  each  program  be  assessed  to 
determine  whether  it  is  achieving  its  stated  goal,  the  improvement  of  flightcrew 
coordination  and  performance.  Each  organization  should  organize  a  systematic 
assessment  program  both  as  a  means  of  tracking  the  effects  of  its  training  program  and 
as  a  means  of  making  continuous  improvements  and  defining  critical  topics  for  recurrent 
training.  Assessment  of  the  training  program  should  involve  both  observation  of  training 
process  and  self-reports  of  participants  using  standard  survey  methods. 

The  emphasis  in  this  assessment  process  should  be  on  crew  performance.  The 
major  CRM  areas  of  assessment  should  include  communications  processes  and  decision 
behavior,  team  building  and  maintenance,  workload  management  and  situational 
awareness  along  with  traditional  technical  proficiency.  An  additional  function  of  such 
assessment  is  to  determine  the  impact  of  CRM  training  and  organization  wide  trends  in 
crew  performance. 

b.  Reinforcement  and  feedback  are  essential  for  effective  CRM  training  programs. 
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(1 )  Reinforcement  is  essential  to  operational  impact.  Crewmembers  must  receive 
continual  reinforcement  for  the  effective  practice  of  CRM  concepts.  Effective 
reinforcement  requires  providing  information  to  crewmembers  on  their  human  factors 
practices  as  well  as  technical  performance. 

(2)  Specific  feedback  requires  consistent  assessment  of  crew  performance  to 
crewmembers  on  the  human  factors  practices  as  well  as  technical  performance. 
Crewmembers  and  those  involved  in  training  and  evaluation  should  be  able  to  recognize 
effective  and  ineffective  human  factors  behaviors.  This  does  not  imply  that 
crewmembers  should  be  formally  evaluated  and  graded  on  the  practice  of  CRM 
concepts,  but  rather  that  these  issues  should  be  incorporated  in  feedback  and 
reinforcement  of  all  aspects  of  training  and  operations.  The  security  and  confidentiality 
of  assessment  data  should  be  addressed  in  developing  assessment  plans.  (See  the 
FAA's  CRM  Handbook  for  discussion  and  guidelines.) 

The  emphasis  in  this  assessment,  process  should  be  on  crew  performance.  The 
major  CRM  areas  of  assessment  should  include  communications  processes  and  decision 
behavior,  team  building  and  maintenance,  workload  management  and  situational 
awareness,  and  as  well  as  technical  proficiency. 

(3)  An  additional  function  of  such  assessment  is  to  determine  the  impact  of  CRM 
training  and  to  determine  organization  wide  trends  in  crew  performance. 

(4)  For  optimal  assessment,  data  on  crewmember's  CRM  attitudes  and  behavior 
should  be  collected  prior  to  the  awareness  phase  of  CRM  training  and  again  at  intervals 
after  training  to  determine  both  initial  and  enduring  effects  of  t@e  program.  The  goal 
should  be  to  obtain  an  accurate  picture  of  the  state  of  the  organization  before  formal 
adoption  of  this  type  of  training  and  to  continue  to  monitor  the  same  elements  after 
implementation. 

c.  In  summary,  assessment  techniques  should  serve  to: 

(1)  Measure  the  operational  state  of  the  organization. 

(2)  Determine  topics  in  need  of  further  emphasis  or  further  instruction  within  the 
CRM  program. 

(3)  Ensure  that  all  check  airmen  and  instructors  are  well  prepared  and 
standardized. 

12.  THE  CRITICAL  ROLE  OF  CHECK  AIRMEN  AND  INSTRUCTORS. 

a.  The  success  of  any  CRM  training  program  will  ultimately  depend  on  the  skills 
of  the  personnel  responsible  for  administering  the  training  and  observing/measuring  its 
effects.  Thus  it  is  vitally  important  that  CRM  instructors  (course  facilitators),  check 
pilots  (operational  reinforcers),  and  course  designers  (developers),  be  highly  skilled  in 
all  areas  related  to  the  assessment  and  practice  of  CRM.  It  is  important  to  note  that 
these  skills  are  different  from  and  in  addition  to  those  associated  with  traditional  flight 
instruction  and  checking. 

b.  Gaining  proficiency  and  confidence  in  CRM  instruction,  observation,  and 
measurement  requires  special  training  for  instructors  and  check  pilots  in  CRM  training 
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methods  such  as  role-playing  simulations,  systematic  crew-centered  observation, 
administering  LOFT  programs,  and  providing  effective  and  helpful  feedback  to  crews. 

c.  Instructors  and  check  pilots  also  require  additional  training  in  observing  crew 
performance  for  purposes  of  calibrating  and  standardizing  their  assessment  and 
debriefing  skills.  This  will  help  to  ensure  consistency  and  high  quality,  industry 
standards  of  performance. 

d.  In  order  to  provide  maximum  learning  for  all  crewmembers,  instructors, 
simulator  and  line  check  personnel  must  use  every  available  opportunity  to  emphasize 
the  importance  and  use  of  crew  coordination  skills  and  techniques.  This  is 
accomplished  best  by  having  the  crews  examine  their  own  performance  and  behavior, 
with  the  assistance  of  a  trained  instructor  who  can  point  out  both  positive  and  negative 
aspects  of  CRM  performance.  Whenever  highly  effective  examples  of  crew  coordination 
are  observed,  it  is  vital  that  these  positive  behaviors  be  discussed  and  reinforced. 
Debriefing  and  critique  skills  are  important  tools  for  instructors  and  check  pilots  to 
acquire  and  utilize. 

e.  Feedback  from  instructors  and  check  airmen  will  be  most  effective  if  it  reflects 
the  concepts  and  behaviors  that  are  covered  in  the  initial  indoctrination/awareness 
training  and  if  the  feedback  refers  to  specific  behaviors  rather  than  vague  generalities. 
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Appendix  I 

Crew  Performance  Marker  Clusters 
(Italicized  Markers  apply  to  Advanced  Technology  Flight  decks) 

These  behavioral  markers  are  provided  to  assist  organizations  in  program  and  curriculum 
development  and  as  guidelines  for  feedback.  They  not  presented  as  a  means  of 
evaluating  individual  crewmembers. 

A.  Communications  Processes  and  Decision  Behavior 

Briefing  (conduct  and  quality).  The  effective  briefing  will  be  operationally  thorough,  in¬ 
teresting,  and  will  address  coordination,  planning,  and  problems.  [Although  primarily  a 
Captain  responsibility,  other  crewmembers  may  add  significantly  to  planning  and 
definition  of  potential  problem  areas.] 

Inquiry/Advocacy/Assertion.  This  rating  assesses  the  extent  to  which  crewmembers 
advocate  the  course  of  action  they  feel  best,  even  when  it  involves  conflict  and 
disagreements  with  others. 

Crew  self-critique  (decisions  and  actions).  This  item  evaluates  the  extent  to  which 
crewmembers,  conduct  and  participate  in  a  debriefing,  operational  review,  and  critique 
of  activities,  which  includes  the  product,  the  process,  and  the  people  involved.  Critique 
can,  and  should,  occur  during  an  activity,  and/or  after  completion  of  the  activity. 

Conflict  resolution.  If  crewmembers  engage  in  conflict  while  attempting  to  decide  on 
a  course  of  action  or  for  any  other  reason,  the  effectiveness  of  means  used  to  resolve 
the  conflict  and  the  use  of  available  resources  is  rated. 

Communications/Decisions.  This  rating  reflects  the  extent  to  which  free  and  open  com¬ 
munication  is  practiced.  It  includes  providing  necessary  information  at  the  appropriate 
time  (for  example,  initiating  checklists,  alerting  others  to  developing  problems).  Active 
participation  in  decision  making  process  encouraged  and  practiced.  Questioning  of 
actions  and  decisions  is  proper.  Decisions  made  are  clearly  communicated  and 
acknowledged. 

1 .  Establishes  team  concept  and  environment  for  open/interactive  communications 
(e.g.,  calls  for  questions  or  comments,  answers  questions  directly,  listens  with 
patience,  does  not  interrupt  or  "talk  over",  does  not  rush  through  the  briefing, 
makes  eye  contact  as  appropriate). 

2.  Identifies  potential  problems  such  as  weather,  delays,  and  abnormal  system 
operations.  Sets  expectations  for  how  deviations  from  S.O.P.  are  to  be  handled 

3.  Provides  guidelines  for  crew  actions  -  division  of  labor  and  crew  workload  ad¬ 
dressed 

4.  Includes  cabin  crew  as  part  of  team  in  the  briefing,  as  appropriate 

5.  Operational  decisions  are  clearly  stated  to  other  crewmembers  and  acknowledged 
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6.  "Bottom  lines"  are  established  and  communicated  for  safety  of  operations.  The 
"big  picture"  and  the  game  plan  are  shared  within  the  team  including  flight 
attendants  and  others 

7.  Crewmembers  are  encouraged  to  state  their  own  ideas,  opinions,  and 
recommendations 

8.  Crewmembers  speak  up,  and  state  their  information  with  appropriate  persistence, 
until  there  is  some  clear  resolution  and  decision 

9.  Crewmembers  are  encouraged  to  ask  questions  regarding  crew  actions  and 
decisions  and  answers  are  provided  openly  and  non  defensively. 

10.  Critique  is  given  at  appropriate  times,  both  low  and  high  workload,  and  is  made  a 
positive  learning  experience  for  the  whole  crew  --  feedback  is  specific,  objective, 
based  on  observable  behavior,  and  given  constructively. 

1 1 .  Critique  is  accepted  objectively,  and  non  defensively,  deals  with  positive  as  well  as 
negative  aspects  of  crew  performance. 

1  2.  When  conflicts  arise,  the  crew  remains  focused  on  the  problem  or  situation  at  hand. 
Crewmembers  listen  actively  to  ideas  and  opinions  and  admit  mistakes  when 
wrong. 

13.  Assignment  of  blame  is  avoided  -  the  focus  is  on  determining  what  is  right,  not 
who  is  right.  Crewmembers  treated  with  empathy  and  respect.  When  there  is 
time,  crewmembers  explain  "why”  particular  decisions  were  made. 

14.  Establishes  policy  guidelines  for  the  operation  of  automated  systems  (i.e.  when 
system  will  be  disabled,  programming  actions  that  must  be  verbalized  and 
ackno  wledged) 

1 5.  Specifies  PF  and  PNF  duties  and  responsibilities  with  regard  to  automated  systems 

16.  Crewmembers  verbalize  and  acknowledge  entries  and  changes  to  automated 
systems 

1  7.  Crewmembers  question  status  and  programming  of  automated  systems  to  verify 
and  ensure  situational  awareness 
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B.  Team  Building  and  Maintenance 

Leadership,  Followership,  &  Concern  for  tasks.  This  rating  evaluates  the  extent  to 
which  appropriate  leadership  and  followership  is  practiced.  It  reflects  the  extent  to 
which  the  crew  is  concerned  with  the  effective  accomplishment  of  necessary  tasks. 

Interpersonal  relationships/Group  climate.  This  evaluation  reflects  the  quality  of  ob¬ 
served  interpersonal  relationships  among  and  the  overall  dimate  of  the  flightdeck.  This 
is  independent  of  demonstrated  concern  with  accomplishment  of  required  tasks. 

1 .  Coordinates  flightdeck  activities  to  establish  proper  balance  between  authority  and 
assertiveness,  acts  decisively  when  the  situation  requires 

2.  Demonstrates  desire  to  achieve  most  effective  possible  operation 

3.  Ensures  that  group  climate  is  appropriate  to  operational  situation  (i.e.  social 
conversation  in  low  workload  conditions  but  not  high) 

4.  Shows  sensitivity  and  ability  to  adapt  to  other  crewmembers'  personalities  and 
personal  characteristics 

5.  Recognizes  symptoms  of  psychological  stress  and  fatigue  in  self  and  others  (e.g., 
note  when  a  crewmember  is  not  communicating,  and  draw  him/her  back  into  the 
team;  recognize  when  they  are  experiencing  "tunnel  vision",  and  seek  help  from  the 
team) 

6.  "Tone"  in  the  cockpit  is  friendly,  relaxed,  supportive. 

7.  Ensures  that  non-operational  factors  such  as  social  interaction  do  not  interfere  with 
necessary  task  duties 

8.  During  times  of  low  communication,  crewmembers  check  in  with  each  other  to  see 
how  they  are  doing 

9.  Recognizes  and  deals  with  demands  on  resources  posed  by  operation  of  automated 
systems 

10.  Reverts  to  lower  level  of  automation  when  programming  demands  could  reduce 
situational  awareness  or  create  work  overloads 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  C-1 5 


Crew  Resource  Management-  Advisory  Circular  120-5 la  15 

C.  Workload  Management  and  Situational  Awareness 

Preparation/Planning/Vigilance.  This  rating  indicates  the  extent  to  which  crews 
anticipate  contingencies  and  actions  that  may  be  required.  Excellent  crews  are  always 
"ahead  of  the  curve"  while  poor  crews  continually  play  catch  up.  Vigilant  crews  devote 
appropriate  attention  to  required  tasks  and  respond  immediately  to  new  information. 
A  crew  indulging  in  casual  social  conversation  during  periods  of  low  workload  is  not 
necessarily  lacking  in  vigilance  if  flight  duties  are  being  discharged  properly. 

Workload  distribution/Distraction  avoidance.  This  is  a  rating  of  time  and  workload 
management.  It  reflects  how  well  the  crew  managed  to  distribute  the  tasks  and  avoid 
overloading  individuals.  It  also  considers  the  ability  of  the  crew  to  avoid  being 
distracted  from  essential  activities  and  how  work  is  prioritized. 

1 .  Actively  monitors  weather,  aircraft  systems,  instruments  and  ATC  communications, 
sharing  relevant  information  with  the  rest  of  the  crew 

2.  Avoids  "tunnel  vision",  being  aware  of  factors  such  as  stress  that  can  reduce 
vigilance  --  thus,  monitoring  the  performance  of  other  crew  members 


3.  Stays  "ahead  of  curve"  in  preparing  for  expected  or  contingency  situations 
(including  approaches,  weather,  etc.) 

4.  Verbally  insures  that  cockpit  and  cabin  crew  are  aware  of  plans 

5.  Workload  distribution  is  clearly  communicated  and  acknowledged  to  maximize 
efficiency. 

6.  Ensures  that  secondary  operational  tasks  (i.e.  dealing  with  passenger  needs, 
company  communications)  are  prioritized  so  as  to  allow  sufficient  resources  for 
dealing  effectively  with  primary  flight  duties 

7.  Recognizes  and  reports  overloads  in  self  and  others 

8.  Plans  for  sufficient  time  prior  to  maneuvers  for  programming  of  automated  systems 

9.  Ensures  that  all  crewmembers  are  aware  of  status  and  changes  in  automated 
systems  parameters 

10.  Crewmembers  recognize  potential  distractions  posed  by  automation  and  take 
appropriate  preventive  action,  including  reversion  to  lower  levels  of  automation 

D.  Overall  Technical  proficiency 

This  is  a  rating  of  how  well  the  crew  as  a  unit  discharged  the  technical  aspects  of  the 
flight.  It  reflects  awareness  that  a  high  degree  of  technical  proficiency  is  essential  for 
safe  and  efficient  operations.  Demonstrated  mastery  of  CRM  concepts  cannot 
overcome  a  lack  of  proficiency.  Similarly,  high  technical  proficiency  cannot  guarantee 
safe  operations  in  the  absence  of  effective  crew  coordination.  This  rating  can  be 
thought  of  as  a  more  fine  grained  evaluation  of  the  technical  performance  of  a  crew 
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than  the  typical  "S"  or  "U"  employed  in  a  Line  Check  or  other  evaluation.  A  "5" 

represents  an  unusual  demonstration  of  proficiency  while  a  "1 "  would  reflect  seriously 

substandard  behavior.  The  typical  well  qualified  crew  would  receive  a  "3". 

1 .  Adheres  to  regulations  and  Air  Traffic  Control  requirements,  and  follows  company 
established  procedures  including  checklist  management  and  standard  callouts. 

2.  Observes  and  effectively  manages  sterile  cockpit  environment. 

3.  Demonstrates  a  high  level  of  basic  (stick  and  rudder)  flying  skills  . 

4.  Required  briefings  include  all  pertinent  safety  and  operational  issues  as  defined  in 
relevant  manuals 

5.  Demonstrates  knowledge  of  aircraft  systems  and  normal,  abnormal,  and  emergency 
procedures 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


APPENDIX  D: 


Smith-Hamman  Briefings 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-iii 


uniTGD  AiRLines 


Human  Factors  and  Observable  Behaviors 
Crew  Performance  Workbook  for 
United  Airlines  Flight  723  LAS/LAX 


Page  D-iv 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


(/) 

t— 

a>  c 

p  ° 

t  X) 

CD  © 

E  40 

>  © 

>  X3 

v-  C 

a  © 

tr  w 

-c  o 
O)  sz 
==  a 

o  2 

©  2  . 
C  >  OT 
O  </)  C 

~  -p.  o 

£  ®  © 
©  5  ,2> 

o§| 
2  £  £ 

gtj  o 

•2  t  « 
c  o  w 

£  ©  .9* 

°  s  I 
©  ~ 
is  E  8 


</)  o  • 

Q-  (D  *♦— 
O  CD.® 
■8  2  £ 
o-8E 

ill 

O  ©  c 

.9  «  c 

9  w  o 

CO  — 

■o  ^  © 

3  E  3 
o  ©  © 

■5  E  =o 
«  >  ® 
§  2fi 

■c  o  W 

®  2  o 

©  |  g 

f  E  o 

*  O  © 
>s  O 

«  £ 
®  E  2 

X)  3  © 

S  §.2 

© 


2 

o 


mow 

©OF 


X>  s 

o  3 


« 


g  © 


© 


®  § 


© 

w 
o 
T3 
3 

L. 

T"  w 
c  £ 

5  H 
©  u. 

£  O 


©  « 
€  S 


© 

xr 


o 

© 

-q 

'© 

"5 

o 


© 

E 

© 

« 

© 


« 

C  -D 
^  —I  o  © 

O  ®  co  ® 

<=  «  3  "D 

XJ  -X  -ts  '© 
—  ii'  ©  c 

o  ®  ©  g 

©  .9  2  a> 

-Q 


^  CD 
try  v_ 

I?  £ 

§<1 

®  ^  © 

®  c  $ 

S  E  2^ 

£  o  O 

.  CL 

©.If 

©  ®  - 
$  ©  ® 

°  —  £ 

x  ®  XJ 
© 

4=  « 


o 
c 

2 
a 
o 

x: 
© 

^  o 
c  o 

2  S 

®  O 
? 
w 


© 

x: 

CD 

n 

X3 

Ul_ 

O 

E 

L- 

o 

x: 

O) 

u 

o 

© 

© 

sz 

_j 

© 

x: 

c 

1c 

CD 

in 

•— 

r~ 

© 

o 

"O 

C"  ' 

O 

© 

L _ 

O 

-*-» 

c 

© 

© 

"x: 

$ 

o 

© 

5 

© 

"O 

CD 

E 

© 

X- 

x= 

■*—* 

C 

m 

© 

x: 


2  'o 
o  =*  ® 
5 !  o 

"©  ©  © 
♦2  C  X) 

•g  2 

o  10  O 
©0)0 
©  c  5 

2  ’£  « 
®  ©  Z. 
«  ©  I— 

«  XJ 

fS  -g 

o  ©  5 
©  CL  O 

©  E  *- 

jS  ©  '© 

Q-  ©  £ 

€  «  g 

©  ©  *3 

LLI  <  To 


w 

© 

•a 

© 


c 

© 

> 

© 


!c 

$  ~ 
©  w 

i  o  2. 

O  XJ  I— 

l.sg 

© 

CD 
C 

X  ©  © 

©  c 
—  X) 
©  © 
x:  XJ 


®  £ 
1 3 

x: 
©  o 

t —  rrl 


© 

© 


C 

© 


CD  C 
©  O 

4-  C  — 

Co® 

—  c  ® 

.♦SOCK 

>  -*-  o  © 
Z  >»  c  c 
£  -•£  ®  - 
,g>  -i= 

5=  © 

—  © 


CL  3 
©  © 

“  It 


si-i&g 

«  .E  ©  F  X 
©  w  2  fc  — 1 
JC  o  - 

o-  E 
><  ^ 


© 

© 
k_ 

© 

2 

l—  C 


®  o  ® 
c  .9  x: 

©  x:  ■— 
Q-  5  o 
x  ^  ♦- 

;  o  ^ 


§  I  £  5 
£  5  «  § 

ra  I  E 


© 

© 

©  sz 

w 

© 

>,  ^ 

c 

.-tr  © 

o 

§  | 

© 

|  1 

E 

E  w 

o 

8  10 

E 

o  ® 

© 

h=  zc 

x: 

c.  ° 

.1  E 

>— 

9 

5 

"  °  2 
©?o 

—  ©  .© 


•O 
c  ♦- 
w  c 

si 

2  w 

<f  © 

CL  :E 


c: 

o 

'© 

w 

© 

V-. 

CL 

E 


«f  ©  _ 

-  F  © 


©  © 
E  2 

-  T5 
©  © 

:i  © 

£  2 
f~  o 


© 

> 

e  5 

3  2 

«-•  © 
3  3 
O  2 
=  > 
*-  -O 
o  c 

©  © 


3  O 

c 

c  "O 
E  © 


>>  o 
©  B 
©  5 

w  c 
£  F 

=  3  «  © 

sz  ©  XJ 

go© 

C  © 

©  © 

E  ® 

©  -o 

©  nJ 
© 

•O 

_  © 

5  % 

«  ~ 


5  ® 
©  © 


o 
♦=  c 
©  = 
W  *5 

©  5 

®  -D 


.8  'S  h 


CL  C 
© 


© 
©  > 
—  © 
©  XJ 

o  o 
T5  tL 
3  c* 

©  c 

i? 

X  8 


fi 

o  8 

-i  © 

P  *5 
■o 

S  o 
®  >» 
£  2 

£  Q. 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-1 


05 

CD 

JZ 

to 

x> 

i  ! 

o  -9 

Q.  w 

to 

.-t;  o 

6  °- 

■—  © 


—  w 

S  E 

D)  ® 

S  -Q 

2  2 

n  OL 


©  © 
b;  jc 
3  - 

•c  10 

W 

a. 

«  © 


© 

5 


TJ 

© 

t- 

cl 

I 

H 

x 

a 


c 

© 

■4-* 

O 

CL 


© 

o 

c 

© 

E 

*_ 

o 

t: 

© 

Ql 

© 

.a 

<o 

to 

o 

CL 

IL 

o 

o 

CD 

>> 

© 

a 

4-« 

O 

X 


© 


rt 

o 

o 

c 


to 

S’ 

© 

TJ 

© 

fc— 

2 

© 

> 

© 

to 

to 

© 


CO 

H- 

< 


© 

© 

© 

c 


© 

> 

© 

> 

c 

o 

O 


CJ 


in 

c\j 

>» 

© 

$ 

c 

2 


2 

3 

■o 

© 

O 

o 

k- 

a. 

y~ 


IL 

U. 

O 

8  i 

2  I 

a  uj 


PHASE  OF  FLIGHT:  PREFLIGHT  AND  GROUND  OPERATIONS 

BRIEFING 


Page  D-2 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


o 

Z.: 

* 

< 

2 

z 

o 

(O 

o 

LU 

D 

Z 

o 

< 

o 

z 

=> 

s 

s 

o 

o 


Jt  -D 

«  5  "o 

a  »  o 
Q  ©  TJ 

£  > 

-O  O  o 


o  5.  . 

•  «  >. 
*  a  ® 

o  5  in 
Z,  ©  c 

_  cr>  m 

is? 

c  2 
-  «  TJ 
©  ©  C 

c  c  « 
o  O  >, 

©  .2  C 

o  o  o 

D  CD  Q. 

cr  t>  o 


*1  z 

©  S’ 

-Q  »- 

E 
a 
E 
t 
a 

t- 

o 


-Q 

E 

© 

E 

i 

© 

o 


e  § 


ra  XI 

-  S 

2  E 
T5  E 
©  o 
o>  o 
(0  © 


«  c 

C  Q. 
©  o 
X)  - 
c  •* 

£  ® 
®  © 

E  5 

1  i 

O  O 


^  "8 
to  cn 
a.  ro 

1  2 

5  * 

<  © 


c 

.2  . 

§1 

o  K 

c  ® 
©  © 
©  j= 

S  5 

£  2 

5? 

xi  a 

S1? 

CT>  M 
£  c 

5  s* 

~  o 

c  ® 

o  £ 


©  ■§ 

(Q 

«  © 

X  * 
a  O 


C 

o 

TJ  ~ 
c  w 

*  © 
TJ  Q. 
«  O 


tO  JZ 
C  OJ 
to  — 

Q-  Q> 

©  C 

E  TJ 
c  «.  ra  3 
»  o  CJI  o 

5  >•  ®  £ 

£  £  -  e  « 

sals® 

a  w  pi  x 

!=  O  r  ©  o 

ir  —  TJ 
2  C  C 

o  "Z.  m 
Q.  =  2 
o»  *  c 

5  -g  ■§ 
:  2  c 

•  ra  ® 


£ 

©  O. 

c  ° 

t  © 

O  CL 
2  © 
d  x: 
o  — 


©  © 


v>  © 
1 1 
I  § 

s  i 

o  E 

CD  O 


Of—©© 


^  to 
©  © 
©  © 
TJ  © 

3  2 

O  2 

S  °“ 

4=  ® 

o  5 

£  ^ 
?  O 

!! 

«  O. 


Involved. 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-3 


$  .2 

K  a 

2  o 

CD  c 
:=  T3 

s  s 

■*-  nt 
m  ® 
o 
®  c 

jO  CD 

If 

c  0, 

£2 

.oj  to 

— -  k. 

T 

SS 

E  CD 

w  S 


-If 

5  S  Q. 

£  «  8 

C  X>  O 

o  ®  nj 

+■*  *=  I- 

2  3  ° 
<D  — 

q.  .a 

O  |  e 

T3  O  ® 

c  So 

3  <  Ql 

O 


JO  O 


to  I 

<  £ 


Page  D-4 


Workshop  on  Integrated  Crew  Resource  Managemen 


p  ° 

®  n 

£  . a 

MOW 


"5  °  — 
S.H  ® 
£  2  £ 
Q.  ^  ™ 

£>.} 
.  O  . 
A  C  « 
><2>0 
c  o>jz 
3  r  O 
0—0 
c  0 
005. 

Tj“a 

2  5;® 

2  °  a, 

«  ®  -E 

5-S.-5 
OT  S  S. 


If 

«  • 

•  > 

3  _  « 

*•  o  o 
CO  2 

I  5  l 


%  8  . 
5?  “ 

£  ®  o 
i:  "S  o 
«  5  jP 

•og® 

*D  7:  O 

©  E  N 

III 

1  |  S 

£  8  E 


CO  >* 
o 
> 


i  s  > 

c  *c  o 


«  S  c 

o  m  5 
~  c  J 
8  o  « 

3  3  <0 
O’  (0  C 

«  -  o 

^  o  *- 
©  _  o 
U  ?3 

E  £  2Z 

O  c  « 

S  co  2 
00® 
*?  *-  c 
O  0.0 


J5  TJ  w_ 

m  ®  © 

CMC 

3  3  £ 

«  TS  u 

C  ®  « 
UJ  n  a 


^  X 

t2 


PHASE  OF  FLIGHT  -  Takeoff 


Workshop  on  Integrated  Crew  Resources  Management  (CRM)  Page  D-5 


O 

O 

O 

z 

6 

o 

v>  . 
5  © 
©  > 

b  3 
® 
®  c 

5  © 
®  E 

©  £ 

3  O 

(0  ^ 


CO 


o  ® 

cr  *■’ 

©  o 


O) 

® 

X) 


c 

o 

b 

© 


5  © 

E  "o 
©  (0 
5  o 

1,1 

II 

V)  © 

w*  ® 
£ 
c  © 
E  = 
©  ® 

> 

© 


:  5 


« 

© 


cr 
o 

i_  '© 

©  •© 
®  © 
2  X> 


© 


© 

> 

UJ 


LL 

LL 

o 

LU 

* 

< 

I- 

■  m 

H 

X 

g 

LL 

LL 

O 

LU 

CO 

< 

X 

CL 


o 

E 

Ui 

t— 

o 

2 

< 

2 

hZ 

o  c 

Ui 

tr  -c 

£ 

3  i 
o«* 

4-  C 

< 

non 

5 

a 

2 

®  ©  > 

5J  « 

o 

C 

>  ©  o 

2 

< 

UJ 

X 

O  .  TJ 
co  © 

o 

2 

CD 

CL 

11  S  s' 

5* 

-i 

X 

(0 

T)  ©  -  *- 

4-  x}  «n  “ 

-c  t!  «  S* 

c 

O)  a  o  cr 

3 

UJ 

Sn.g£. 

CD 

a 

«  e  >  c 

£ 

< 

< 

UJ 

©  CL  ^  ^ 

®  -C  ©  a 

c  9)  m  3 

^  co  ^ 

£  -O  «  ® 

°  2  T3  « 

O  w  c  ® 

O  ©  co  — 

Page  D-6 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


o> 

> 

a> 

© 

*-■ 

55 

TJ 

<d 

E 

v_ 

a ) 
c 

TJ 

d 

CO 

n 

E 

O 

© 


> 

CO 

TJ 

CD 

a. 

o 

CD 

> 

© 

T> 

sz 

o 

Id 

$ 

E 

© 

X) 

2 

CL 

O 

d 

o 

a) 

CL 

E 

o 

CJ 

TJ 

d 

CO 

c 

o 

V-* 

CO 

3 

c 

ci 

o 

O 


w 

a? 


© 

-C 


o 

CD 

> 

CO 

c 

© 

3 

CO 

2 

■d 

o 

Id 

5 

d 

o 

T3 

CD 

CO 

k_ 

© 

d 

CO 

'co 

CO 

o 

«4— 

o 

CO 

© 

k- 

© 

© 

sz 

<z 

>k 

> 

TJ 

© 

© 

> 

T3 

© 

> 

d 

o 

O 


(f) 

3 


3 

O 

-O 

E 


© 

sz 


CO 

t: 

o 

CL 

© 

© 

o 

c 

© 

3 

JQ 


O 

O 

O 

■*— » 
ro 

p 

© 

X 

CD 

C 

‘© 

x> 

c 

© 

o 

k_ 

'© 

© 

SI 

d 

CO 

3 

w 

© 

k_ 

JZ 

o 

sz 

£ 

E 

© 

</) 

>> 

</) 

o 


>» 

© 

3 

CO 


o 

© 

© 

E 

3 

E 

c 

E 

CD 

C 

© 

© 

E 


d 
© 
E 

CD 
© 
W 

JZ 
CD 

© 

sz 

o 

d 

© 

© 

l _ 

CD 
© 

C 
© 
o 

_  o 

jtr  3 


w 

E 

© 

X> 

O 

a. 

© 

o 

d 

© 

E 

o 

■StJ 

cl  a> 

d  © 


3  - 

*“  X) 
>>  d 
© 

E  o 

jz  © 

.2  d 

x:  © 

z* 

c  © 

W  -§ 

SZ  3 

g) 

X  © 


© 

c 


a 

JZ 

$ 

© 

c 

© 

> 

© 

© 

© 

*v_ 

© 

> 


c 

I 

h- 
X 
t D 


© 


a 


© 


O 

LU 

< { ) 

< 

X 

a 


w 
-* — * 
d 
© 
> 
LU 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


u. 

UL 

O 


LU 

> 

LU 


LU 

H 

< 

Q 

LU 

2 

DC 

LU 

H 

2 

a 


< 

m 


< 
i — 


x 

g 

IL 

U. 

O 


LU 

if) 

< 

X 

CL 


Page  D-7 


Page  D-8 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-9 


Initiates  and/or  participates  In  operational 
review  which  Includes  the  outcome  (pro¬ 
duct),  the  process  and  the  people  Involved. 


Page  D- 1 0 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


© 

2  5 

o  © 

o  -o 
2  ® 


(0 


Ef 

-5  ©  d 
0^0) 
q-£  c 
S  $  15 
.9  ^  ra 

1-2  c 

XJ  0)  o 

©  c  E 

r-  —  in 

S  2  o 

o,  3  ^ 

O)  cr  n 

5|f 

1  5  o 

.c  c 

$  o  ® 

2  ^  3 

b  ?  € 

-  ~  13 


c 

0) 

a 


c 

a) 


>  CD 


w 

to  ®  o> 
©  CO  C 


•a 


a.  o 
o  sz 
~  o 


CD 

£  « 

©  CO 

CO 


Ql 

o 

CO 


c 

© 

o 

CO 

a> 

Q 

i 

t- 

X 

o 


.9  o  r 
>-  -c  a) 
D.  «  .j 


© 


a> 

o 

<n 

a> 

x> 

Jj 

a>  o 

^  a. 

2  I 

P  b 

O  x: 
sz  •*- 
«  x: 

I  'I 

b  CD 
T"  O 
ffi  c 
-C  CO 


CL 

E 

O 

5 

o 


CD 

> 

CD 


2 

CD 


CO 

2  _ 

aj  co 


ca 

o 


o 

c 


XJ  ^ 
CD  40 

^2 


<D 

U 

CD 

-O 

P 

zj 

o 

SZ 

CO 

CD 

o 

cz 

2 

CO 

o 

o 


a) 

o 

c 

co 


t> 

CD 

©  B- 

o  >< 
c  ® 
CO  o 

CO  CD 

_a>  n 
o  g 
■D  & 

3  2 
CL  5 

e  r 

O  CD 
O  > 

-o  ® 

CD  ~ 

cl  x: 

CL  D. 
CD  ;£ 


CD 

to 

»= 
o 
<n 
CD 
> 

40  CD 

co  o 

b  s 

CO 

©  =2 

£  o 

©  Q- 
© 


«  c 

CO  ‘co 

-1 

w  CO 

«  s 

c 

©  c 

>  © 

rn  CJ 
D)  w 

in  © 

-  tj 
S  CO 
c  £j 
<0  f'* 
x> 


co 


© 


o 
tr 
© 
CL 

C  c 
®  © 
9  o 
®  w 

©  CD 

Q  xj 


© 

o  c 

C? 
©  - 
CJ  CD 

in  ~ 

CD  CL 

XJ  E 

CD  © 
XT  X 
tz  CD 

C  C 
©  CO 
>  w 
5  < 


<  > 
o  ^ 

©  5 

n 

v- 

x:  o 

OJ  ■*“ 

®  © 

©  J- 
C  © 

o  © 

?S 

©  XJ 

Z  P 

O)  o 
ip:  -C 
^  CO 

co  4^ 
c  2 
.2  b 

S'3 

CO  CD 


C  O 

~  © 
©  n 

Q§ 

©  N 

is 

O  5 


CO 

■4 

C. 

O 

o 

]g 

3 

o 

x: 

tn 

X) 

cr 

© 

CD 

O 

X) 

c 

© 

© 

x: 

o  | 
©  cO 

So. 

si 

XJ  © 

CE  ^ 

y-  © 

>  r 

tr 

1  ^ 

E?  o 

if 

CD  -Q 
XJ  CO 

in 


o 


5  8- 


,  © 
w  £ 
X  © 

3  © 

— 1  o 

©  P 
I  O 

to  w 

©  CD 

5  Si 

©  ^ 


H 


© 


U. 


O 

tu 

(/} 

< 

X 

CL 


in 
-♦— * 
Cl 
(J 

UJ 


PHASE  OF  FLIGHT:  DESCENT 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-1 1 


1X1 

o 

z 

< 

z 

111 


< 

2 

o 

z 

< 

a 

z 

a 

-X 

ZD 

m 

5 

< 


1 

e 

a> 

r 

o 

c 

o 

c 

o 

■O  *1 
c  « 

2  3 
o 

E  -* 
o  2 

e  " 

=  t 

■3'S 

©  © 
a  c 


^  -c 

L.  *"* 

o  c 
-c  © 

3  f 

c  ^ 
mo© 
^  ©  > 
-  2  •» 
5  is  s 

s  ®  ^ 

"  “  - 

o 


o 

^  © 


©  ® 

1  a » 

O,  a  2  £ 

=  S-  ®  "5 
«  E  >  f 

2  a- c  £ 
“■awe 
—  —  2  ° 
1  H  s  = 

*»  C  — 

O  •  a  g 


—  cn 

8  ® 

?¥ 

°  I  - 

O  O  2 
Q.  £  — 
0=2 
sal 

o£  ? 

B-  ®  > 

1.3,1 

®  3s  « 

©  4-  <® 

•  X3  C 
N  c  © 

c  O  -C 

O)  »  5 

o  o  £ 
o  ^  n 
EC  ©  C 


2 

c  © 

i  ; 
?.i 

“  "  c 

£  5  “ 


»  c 

E  5 
-o  J 
"o  a> 
c  N 
•  c 
cn  cn 
c  o 
:=  o 

l  2 

§  § 
E 
E 

°  .c 
o 


o 

<n 

> 

"ffi 

c 

c 

3 

»! 
—  s 

o  — 
C  © 


O  8. 


E 

.  $ 


© 

r-  « 
•k 

s  i 

0>  OJ 

E  E 

•  2 

-O  _Q 

E  ® 

Q.  XI 
—  C 

JH  ® 
E 
© 


T3 

© 

S  10 

2  c 
-C  © 

JZ 

2  © 
© 

c  o 
o  — 

75  2 

TS  ®. 

Id 
o  « 
© 


2  3  <5$ 


O  © 

a.  x> 

« 

• —  © 
—  SZ 

c  <0 
©  © 
2  t 


|  | 

c  5 

2  > 

a.  © 

o  X) 


c 

© 

.c 

i 

r  cn 

a  o  c 

5  »  E 
°  E  E 

©  ©  O) 

C  «  £ 

©  «  °: 
XI  X3  X) 

3  ©  © 
cn  ©  .a 

s-i  I 

3  1  1 
a.  ®  j; 

©  *o  — 
©  = 
-c  c  J 
U)  O 

X3  (0  o 
2  ©  © 
<n  Q_  >, 
Ul  o  w 


o 
z 
z 
< 
2  ; 
z 
o 

tn 

o 

ui 

Q 

Z 

o 

i— 

< 

g 

z 

ZD 


o 

o 


xe  -o 
tn  c 
a  (s 
o  « 

i  | 

o>  <J 

e  ; 

3  | 
o  5 

©  O) 

8  - 

5  P 

8  S> 
8 


2 


© 

Q. 

O 

S 

2 

> 

o 

k- 

a. 

© 


|S 

xi  c 

C  © 


E  " 
©  5 
E  ^ 
<  «* 
m  © 

4-  *1 

o 


?  15 
E  c 

.2  o 
©  c 

O  T> 

9  C 

tj  m 


ZZ  £ 
©  T3 

-  S 

2  E 

?  I 

a>  o 

C  8 

^  s 

®  en 

e  o 

<S  c 

E  o. 
2  °. 
I  S 

®  ffl 

E  O 

i  ? 

O  o 


i.:, 


,  a  II 

A“Sj 


o  -S  2 

3.  P  O 

S  8.3 
?  2  :S  | 
5  2  2 ! 

Q.  a.!i;  ; 

l  e  S ; 

™  s-  «J 


8. 


E 

o 

© 


X)  c  — 

E  2  E  . 
®  n  ®  5 
E  c  £  2 

I  1  =  1 
OS  Is 


X3 

© 

> 

8 

e 


c 

& 


£  § 
E  2 

®  a 

I  i 
®  2 
d  £ 


TJ 

C 

ra 

x; 

«  ~~ 
£  a 
a  c 
E  2 

£  a 
*  ~ 

£  « 

:  • 
£.I 

5  -§ 

4_  V. 

a  Q. 
©  © 

5  - 

1  § 
o 
« 
c 
© 

X  O 

^  .2 


“o  cn 
c  c 
a  o 
©  > 
<o  > 
®  c 

X3  © 

2  1 
>*  © 
©  ® 
_  © 
z  z 

c  E 
©  ~ 
©  E 

=  "O 

C  * 

%  C 


I 


c  c 
E  o 

I  i 

O  o 


a  . 

1  - 
*-  © 

2  > 

£  I! 

I! 

If 

s* 
—  © 
^  © 
o  XI 
^  XI 


cn  X> 

f  I 

O  c 


^  2 


© 

-C  T3 

H  © 

.  © 

s  ^ 

o  5  £ 

c  p  © 
®  ©  c 
TJ  Q_  © 
®  O  Q. 
©  o  © 

5  ^  i 

®  ®  s> 

5s; 


■8 

3 


—  D 


I  ^ 

IZ 
o 
CO 

;  O  ; 


z 

E  n- 

E  o» 


jj 


-  the  ;ea>r)  Includinc  flisht  aMendan;«;  I  I  I  actions  that  must  be  verba'lzed  and 

;  others.  I  I  I  acknowledged). 


Page  D-12 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


.  <0 

CD 

£  = 
CD  <D 
o  > 
co  c- 
<d  cd 
■o  c 

§  i 

eg 

9?  -*= 

>  *- 

o  w 

cd  o 

S.I 

O  (p 

S  o 
E  cl 

£  ® 
2  J= 

1° 

o  -c 

V  CD 

o-° 

5  2 

-C  CD 
cn  a 
£  c 

S'  c 

CD 
> 


CD 

-O 


C 

CD 

O 

co 

a> 

Q 


X 

(3 


£ 

CD 

o 

ra' 


O) 

c 

a> 

•c 

r» 

xr 

o 

CO 

2 

CL 

CL 

CO 

CD 

Q) 

CL 

E 

g 

CO 

CD 

4-* 

ra 

w 

co 

CD 

o 

CD 

c 

T3 

3 

O 

-C 

co 

-C 

o 

Ic 

5 

CD 

2 

o 

c 

CD 

CD 

~o 


S' 

C 

CD 

3 

cr 

2 

»»— 

CD 

C 

o 

_g 

"co 

co 

o 

CL 

CO 

o 

£  ■D 
O  co 

11 
*  § 
li? 

CD 

® 
CO  CO 
CD  CO 

b  2 

2  C 

®  2 

O)  C 
CD  ® 
■O  ~ 
Z,  TJ 
TJ  CD 

•3  b 

O  CD 

OT  ® 
CO 


o 

a> 

07  g> 

c  .E 

XX 

x: 

5 

ca 

ca  cd 

© 

xr  x) 

a] 

p 

X 

«  — 

V— 

ca 

3 

5  CD 
c  _c 

a> 

© 

2  r 

-C 

xz 

H 

h- 

<  2 

CJ 


U. 

O 

UJ 

(0 

< 

X 

a. 


> 

UJ 


PHASE  OF  FLIGHT:  FINAL  DESCENT 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D  13 


Page  D-14 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


>* 

< 

£ 

z 

p 

& 


>< 

PQ 

o 

w 

Ph 

< 

CV, 

w 

es 


K 

H 


CO 


> 

w 

V 


ex 

ID 


WILLIAM  HAMMAN 


CRM  INTEGRATION 


INSTRUCTIONAL  EMPHASIS 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-17 


g  C 

o  o 

•rj  • 

(A  "4-> 
(A  <J 

S  <u 

O  X 
u  «*■* 

<u  ^ 


.2  « 
•*-1  *3 


(A 

CA 

QJ 

s 

O) 

M 


£ 

rs 


•«  • « 

U  W 


•  • 

c/) 

r- 

o 


o 


o 

Ut 

o-< 

e 

o 

u 


00 

ns 

M 

73 

VM 


c 

o 


•  • 
Q 


•  • 


Page  D-18 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-1 9 


IH^i 

."pfc 

‘■VOX 

M 

< 


>s  §  ?  V 


r- 

Jh  **h 
U  O* 

3  <u 
w  D-, 
c«  «o 

S-o 

OJ  M 

-h  as 

Xi  V 

O  G 

H  •-* 

n .  <rJ 

^  M 
— H  M 

nj  O) 

-  ■— <  -4-# 

•w 

G  c«v 

QJ  • 

O  <U 


' — '-ZL 

^  G 
<u  OJ 

M  Kf 


5  S  W) 
23  X  .5 

as  73 

<u  S  G 

73  >  O 


G 

o 

—  73 
«  O 

£  3 

01  M 
H  O 

u  S 

M  •* 

O  *n 
^  G 

s  « 

.5  o 

T.  ^ 


•g  ^ 

8>S-d 

S'. §  $ 
T3  55  « 

•»-l  -*-4 

>  >  * 

M  73  73 
P-t  i  rt 


3 

73 

Uh 

Z  G 
^  O 

CD  -H 

<u  73 

•  p*  ro 


«  - 
CX  G 
(/>  rt 


< 


Page  D-20 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


O  w 
weS 

L-J 

P§ 

Sw 

w  Z 

Ph  W 


0*0 

c 

A  n 


«s 

>>2  G 
5  o 
o  .2  *J5 

c  t3  « 

O  4»  O 

a  .a  * 

•p  TJ  O 

60-G  M 

O  «  O 

U  P  *H 

OJ  n  ^ 
*  rt  ^ 

Jh  «J  O 
a;  H 

6*2  G 
C  G  o 
r  «  o 


T3  O 
G  * 

S| 

■P 

G  nJ 
o  •*-' 

u  (JO 
<D  .3 

M  M 

Dm  G 

O)  G 
fv  M 

O  05 

£  C 


Fl.  Proper  and  timely  interpretation  of  propulsion 
information 


i  m 

OwQ 

«  a  o 

a^o 

a  •z  r 
h  a  <-> 
KOB 
o  Z& 
O  w  Q 

Hgo 

£fc2 
<  tG  S 
TJ  << 
hJ  ^  *-h 

(J  W  < 
n  u 


S2^ 

«°3 

gSg' 

9  w  $ 

&-T  ^ 


2  w 

O  > 

f_H 

cdi  u 

CD  W 

S  ST 
o 

M  2 

-  >r*  C/5 


4i 

U 

C 
O)  « 

g  6 

3  O 
O  vh 

s  * 

fs  S 

n 

O)  u 
N  — < 
*2  ro 
3  G 
&0-~ 
O  M> 

s  a 
«  s 


nD 

>  'P 

M 


6 

.S 

V-M 

C 

05  > 


13 

•4-4 

••-* 

O 

D  <* 

12  <N 

a, 

■go 

s§ 

G 

•w4 

<> 

3 

’cJ  o 

X 

03 

•w* 

°co£ 

tH  ►> 

M 

rs 

4) 

>  ■“ 
-2 

t>0 

03  £j  ^2> 

*o 

*  « 
w  rG 

4)  \4-( 

•  • 

.fi 

fi 

•  H 

s?§* 

s-’S 

o  2 

**  a. 
<u  T* 

•4-4 

u 

a 

> 

« i-< 

.5  g  .a 

•2-2  g 

«*■*  H 

o  -S 

4<l 
•  »-« 

C/3 

o 

EG 

G  u  *G 
•h  ji  c 

s 

rH 

cs 

ro 

£5  '*’* 

©  u 
^  4> 


5)  Initiate  right  turn  at  400' 
(5750  AGL) 


Page  D-22 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


i-J 

£>§ 

-si  a 

g3l 

-s  e  s 

Sag 
QzS 
o  w  o 

HC^n 

IS  I 

_J  ^  I— c 

u  w  < 
Sub 
9zz 

'fS  < 

^og 

h£~ 

ssi 


3  ^ 
nj  u 
#->  a» 
6  S3 

•tt  g 

'4-(  O 

eg  u 

■0.53 

U  T3 


T3 

CO 

«**» 

o» 

13 

•  *H 

M 

a 

o 

M 

a, 

O) 

*rt 

fH 

O 

a 

f3 

V 

•H 

*G 

© 

_l 

VM 

cu 

o 

ra 

CM 

lH 

•  -4 

cu 

•44 

•44 

o 

zt 

*d 

w 

>1 

J-l 

•44  . 

<w 

a 

•  H 
M-t 

X 

•  *-H 
•4-1 
»-*4 

ra 

<u 

a 

(0 

n 

o» 

A 

H 

<4 -< 

u 

o 

.a 

rt 

u 

0> 

<44 

44 

J3 

• 

*3 

44 

4> 

X! 

60 

•^H 

CO 

CO 

U 

a 

CO 

On 

o 

ss  g 

O  a 


hJ 

u  - 

-  60 

§’C 

in 

is  «>a 

la  .5  CO 

H  «  O 

>  o  S 

u  «.v 

a)  S  J2 

«*■*  n 
rt  w 

»“<  -j 

-2  fi 

5  £  5 


H  •  £ 

5  -  ^ 

O  A  ^ 

^  2r3 

<u  ,  ' 

y  (V,  T3  WO 

G  J-:  a>  mm  ir1 
3^  D  V  ^ 

g  2  M 

§2.£.2 

<<<u 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-23 


CREW  PERFORMANCE  MARKERS 


Page  D-24 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


9  .O 


d  73 
c>c_ 

•ri  *73 

—i  2 
<«  *13 

|.s 

Iff 

3  *S 

£  ^ 

d  *rt 

2o 

■S£ 

d 

S  8 


s  1 

§  § 

<->  o 

n;  ,-M 

.a  a 

S3  CX 


*°  hJ 

So 


*r'  u  ^ 
°  C  o 

^  J2  + 

JQ  o!> 

3  ^  a 

03  03  t> 
m  o  r  . 


TS 

03 

N  *t3 
75  <y 

d 

03  rti 
•*H  £7 


|  6 
c  3 

Jr!  <-m 

CJ  -pH 
£  w 

T3 

a> 

03 

,  u 

o  *9 

d 

a» 

*X  N 
d  *5 

M  6 

D.  g 
d  a 

d  ” 

03 

ftf 

S 

U 

•w 

S3  a, 

•J-1  o 

S  .2 

•  »-H  -i-3 

•  H 

03 

S 

.5  ° 

sx  c 

O 

o 

a. 

5  o 

.5  s 

03  &fi 

3  S3 

'-M 

M-( 

O 

03 

to  d 

s  s 

on  m 
S3  d 
W  T3 

X 

ra 

• 

tN 

ro 

w 

Mi 

Workshop  on  Integrated  Crew  Resources  Management  (CRM)  Page  D-25 


Page  D-26 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


MISSION  PERFORMANCE  MODEL 


Figure  1 


CREW  PERFORMANCE  MARKERS 

Revision  A  (4/18/91) 


Page  D-28 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  orStandard  Above  Average  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CRTEWPERFORMANCE  MARKERS 

Revision  A  (4/18/91) 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-29 


v» 


co 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  liclow  Performance  Improvement  or  Standard  Above  Averuge  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS 

Revision  A  (4/18/91) 


Page  D-30 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


-r 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Delow  Performance  Improvement  or  Standard  Above  Average  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS 

Revision  A  (4/18/91) 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-31 


H 

Z 

w 

co 

co 

UJ 

CO 

CO 

< 


U 

W 

* 

oc 

< 


e 

•3 


CX 

O 

x— • 

cx 

ex 

rt 

rt 

c 

o 

n 

rt 

E 

£ 

C 

to 


o 

£ 


a 

o 


8 

CX 


o 

«D 

-a 


c 

o 

\3 


•o 

o 

•a 

*> 

s 

cx 

o 

s 

c/1 

o 

.5 


J8 

6 

o 

E 

* 

o 


-5 

o 

9 

■8 

3 


o 

2 

.o 


E 

E 

o 

*o 


cx 

o 


til  rt 

•5  u 

*  ! 
« ^ 

*8  E 

3 


to 

c 

8. 

c/l 

*o 

O 

v~ 

•o 

O 

rt 

o 

-5 

to 

o 

JO 

o 

h* 

’> 

TJ 

c/l 

c 

o 

C 

rt 

O 

CX 

o 

a 

c 

CO 

o 
— 1 

rt 

X> 

2 

i/i _ _ 

o  .rr  — . 


T3 

a 

to 


-o 
to 

-5  o 
o 


o. 

o 

o  -o 

=  £  e 

c/1 

jo 

’3 

to 

XJ  ,*> 

*a  at 

c  o 

V>  t-  rt 

c  2 

-  S  « 

£ 

cx 

o 

> 

*n 

o 

< 

c 

.o 

1 

°  5 

'3  ^ 

2  8 
a.  s 

o  5 

E  o  -5 

2  3  5 
o  3  -S 

o 

E 

* 

o 

1— 

U 

, _ v 

y _ v 

Ol 

m 

V~> 

vO 

2  $ 

a  1 

o  a 

■a  S 

a>  s 

rt  */» 

c 

§  .2 

O 

S  8 

o  X> 

S  -O 


■a 

c 

o 


>* 

a 

o 

a 

u 

H 

< 

U 


S  !p  5 

a  .c  o 

■9  s  .a  3 

V,  2  u  .o 

<w  o  s;  -S  .5 
"  3  5  ■«  S  5 

8  »  1  Q  -5 

5  p  v  \* 
3  o  .2  £.  a 

s  5  i  5  S  ^ 

»  s  s  4.  5-1 

o.  o.  o*  a  6 


*1 
■S  -9 


* 

i 

a 


OC 

UJ 

h- 

U 


a -3 
•25 
s-g 

c  ca  i- 
=3  ui  .2 

E  £  ; 
E  c  -2 

O  £  a, 
US.  O 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  or  Standard  Above  Average  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS 

Revision  A  (4/18/91) 


Page  D-32 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  orStandurd  Above  Average  Significantly  Above 

Expectation*  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS 

Revision  A  (4/18/91) 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-33 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Ttelow  Performance  Improvement  or  Standard  Above  Average  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS 

Revision  A  (4/18/91) 


Page  D-34 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  or  Standard  Above  Average  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS 

Revision  A  (4/18/91) 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-35 


CREW  PERFORMANCE  MARKERS 

Revision  A  (4/18/91) 


Page  D-36 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  or  Standard  Above  Avcruge  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS 

Revision  A  (4/18/91) 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-37 


T* 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  or  Standard  Above  Average  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS 

Revision  A  (4/18/91) 


Page  D-38 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  or  Standard  Above  Average  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS 

Revision  A  (4/18/91) 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-39 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  or  Standard  Above  Averuge  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS 

Revision  A  (4/18/91) 


Page  D-40 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  or  Standard  Above  Average  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS 

Revision  A  (4/18/91) 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-41 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  or  Standard  Above  Average  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS  "TECHNICAL 

Revision  A  (4/18/91) 


Page  D-42 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


•»T 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  or  Standard  Above  Average  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS  "TECHNICAL 

Revision  A  (4/18/91) 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-43 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  or  Standard  Above  Average  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


Page  D-44 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  or  Standard  Above  Average  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS  "TECHNICAL 

Revision  A  (4/18/91) 


Workshop  on  Integrated  Crew  Resources  Management  (CRM) 


Page  D-45 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  or  Standard  Above  Average  Significantly  Above 

Expectations  Needed  Performance  Performance  Standard 


CREW  PERFORMANCE  MARKERS  "TECHNICAL 

Revision  A  (4/18/91) 


Page  D-46 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


V) 


Poor  Performance  Minimally  Acceptable  Satisfactory  Very  Good,  Exceptional  Performance 

Significantly  Below  Performance  Improvement  or  Standard  Above  Average  Significantly  Above 

Expectations  Needed  Performance  ’  Performance  Stundurd 


INSTRUCTIONAL  EMPHASIS 


Page  D-48 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


6.1  7.  Perform 

Communications 


CRM  INTEGRATION 


Page  D-50 


Workshop  on  Integrated  Crew  Resource  Management  (CRM) 


If  the  subtask  involves  lateral  or  vertical  navigation,  key  on  flight 
maneuvers  and  attitude  control,  and  workload  management. 
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I.  NORMAL  MISSION  PROFILE 


Note:  An  asterisk  (*),  which  precedes  any  action  statement  in  the  following  outlines 
denotes  an  item  which  is  generic  to  all  UA  fleets.  Items  which  are  specific  to  the 
operation  of  the  B-737-300/500  do  not  have  an  asterisk  in  front  of  the  statement. 

Note:  To  identify  items  which  are  repetitive,  brackets  containing  outline  paragraph 
numbers  are  placed  after  the  action  statement  to  identify  the  paragraph  in  the  outline 
where  the  item  first  occurred.  Example:  (1.4. 1.1.1] 


1.  *  PREFLIGHT  GROUND  OPERATIONS 

1.1.  *  Perform  dispatch  duties 

1.1.1.  *  Assess  environmental  factors 

1 . 1 . 1 . 1 .  *  Assess  weather 

1 . 1 . 1 .2.  *  Review  terrain/obstacles 

1 .1 .1 .3.  *  Discuss  airport  traffic 

1.1.2.  *  Review/assess  operational  factors 

1 .1 .2.1 .  *  Review  equipment  operating  factors 

1 .1 .2.2.  *  Review  system  operating  factors 

1.1. 2. 3.  *  Assess/comply  with  operational  legality 

1 .1 .2.4.  *  Determine  aircraft  gate  location/flight  status 

1.1. 2. 5.  *  Comply  with  reference  info/guidance 

1.1.3.  *  Review/ Assess  aircraft  maintenance  status 

1 .1 .3.1 .  *  Review  log  history 

1 .1 .3.2.  *  Review  open  items 

1.1. 3. 3.  *  Review  deferred  items 

1.1. 3. 4.  #  Comply  with  reference  info/guidance 

1.1.4.  *  Perform  flight  planning  duties 

1 . 1 .4. 1 .  *  Review  route/altitude 

1.1. 4. 2.  *  Perform  fuel  analysis 

1.1. 4. 3.  *  Perform  weight  analysis 

1 .1 .4.4.  *  File  flight  plan 

1.1. 4. 5.  *  Comply  with  reference  info/guidance 

1.1.5.  *  Perform  communications 

1 . 1 .5. 1 .  *  Perform  cockpit  crew  communications 

1.1. 5. 2.  *  Perform  communications  with  company  (dispatch) 
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1.1. 5.3.  *  Perform  communications  with  maintenance  (Station/SAM) 

1.2.  *  Perform  preflight  duties 

1.2.1.  *  Monitor/ Assess  environmental  factors 

1.2. 1.1.  *  Monitor/ Assess  weather 

1 .2.1.2.  *  .  Monitor/assess  parking  area  congestion  factors 

1 .2.2.  *  Perform  preliminary  cockpit  setup 

1 .2.2.1 .  *  Perform  procedural  flows 

1.2. 2. 2.  Comply  with  reference  information/guidance 

1 .2.2.3.  Enter  FMC/IRS  initial  position 

1.2.3.  #  Perform  aircraft  inspection 

1 .2.3.1.  *  Perform  exterior  inspection  flow 
1 .2:3.2.  *  Perform  interior  inspection  flow 

1 .2.4.  *  Perform  cockpit  preparation 

1 .2.4.1 .  *  (F/O)  Perform  first  officer  procedural  flows 

1 .2.4.2.  (F/O)  Comply  with  reference  information/guidance  (First  Officer) 

1. 2.4.3.  *  (C)  Perform  captain  procedural  flows 

1 .2.4.4.  (C)  Comply  with  reference  info/guidance  (Captain) 

1.2.5.  *  Perform  before  start  preparations 

1 .2.5.1.  *  Perform  procedural  steps 

1.2. 5. 2.  Assess/Enter/Set  FMC/MCP  preflight  data 

1.2. 5. 3.  Comply  with  reference  info/guidance 

1.2. 5.4.  *  Perform  before  start  checklist 

1 .2.6.  *  Perform  communications 

1 .2.6.1.  *  Perform  communications  with  ATC 

1 .2.6.2.  *  Perform  cockpit  crew  communications 

1.2. 6. 3.  *  Perform  communications  with  cabin  crew 

1 .2.6.4.  *  Perform  communications  with  company  (ramp) 

1 .2.6.5.  *  Perform  communications  with  maintenance  (sta) 

1 .2.6.6.  *  Perform  communications  with  passengers 

1.2. 6. 7.  #  Input/Receive  ACARS  data 

1.3.  *  Perform  start/pre-taxi 

1.3.1.  *  Monitor/assess  environmental  factors 

1 .3. 1 . 1 .  *  Monitor  weather 

1 .3.1.2.  *  Monitor  airport  traffic 

1 .3.1.3.  *  Monitor/assess  position/obstacles 

1.3.2.  *  Perform  pushback 
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1 .3.2.1.  *  Perform  procedural  flows/steps 

1.3. 2. 2.  #  Comply  with  reference  info/guidance 

1.3. 2. 3.  *  Perform  before  pushback/taxi  checklist 

1 .3.2.4.  *  Monitor  pushback  activities/progress 

1.3.3.  *  Reconfigure/Check  systems  for  pre-start 

1 .3.3.1 .  *  Perform  procedural  steps 

1.3. 3. 2.  Comply  with  reference  info/guidance 

1 .3.4.  *  Start  engines 

1. 3.4.1.  *  Perform  procedural  steps 

1 .3.4.2.  Comply  with  reference  info/guidance 

1.3.4. 3.  *  Assess  start  abort  decision  factors 

1 .3.5.  *  Reconfigure/Check  systems  post  start 

1 .3.5.1 .  *  Perform  procedural  flows 

1.3. 5. 2.  Comply  with  reference  info/guidance 

1 .3.5.3.  *  Perform  after  start  checklist 

1.3.6.  *  Reconfigure  check  aircraft 

1 .3.6.1 .  *  Perform  procedural  steps 

1.3.7.  *  Perform  communications 

1 .3.7.1.  *  Perform  communications  with  ATC 

1 .3.7.2.  *  Perform  cockpit  crew  communications 

1.3. 7. 3.  *  Perform  communications  with  company  (ramp) 

1 .3.7.4.  *  Perform  communications  with  ground  personnel 

1 .4.  *  Perform  taxi 

1.4.1.  *  Monitor/assess  environmental  factors 

1.4. 1.1.  *  Monitor/assess  weather 

1.4. 1.2.  *  Monitor/maintain  position/obstacle  awareness 

1.4.2.  *  Assess  operational  factors 

1 .4.2.1 .  #  Assess/comply  with  takeoff  legality 

1.4. 2. 2.  *  Assess  special  performance  takeoff  requirements 

1 .4.2.3.  *  Assess/respond  to  mission  delay  factors 

1 .4.3.  *  Reconfigure/check  systems 

1. 4.3.1.  *  Perform  procedural  flows/steps 

1.4. 3. 2.  Comply  with  reference  info  guidance 

1 .4.3.3.  Assess/set/enter  FMC/MCP  data 

1.4. 3. 4.  *  Perform  before  takeoff  checklist  (down  to  "line") 

1 .4.4.  *  Reconfigure/Check  aircraft 

1.4. 4.1.  *  Perform  procedural  steps 

1.4. 4. 2.  Comply  with  reference  info/guidance 
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1.4.5.  *  Maneuver  aircraft 

1 .4.5.1 .  *  Perform  thrust  management 

1.4. 5. 2.  #  Perform  brake  management 

1.4. 5. 3.  *  Monitor/control  directional  steering 

1.4. 5.4.  Comply  with  reference  info/guidance 

1.4.6.  *  Perform  airport  navigation 

1 .4.6.1 .  *  Monitor  airport  ground  traffic 

1 .4.6.2.  *  Identify  taxi  route 

1 .4.6.3.  *  Review/assess  airport  chart 

1.4. 6. 4.  *  Assess/comply  with  ATC  instructions 

1.4.7.  #  Perform  Communications 

1.4. 7.1.  *  Perform  communication  with  ATC 

1 .4.7.2.  *  Perform  cockpit  crew  communication 

1 .4.7.3.  *  Perform  communication  with  cabin  crew 

1 .4.7.4.  *  Perform  communication  with  company  (ramp/dispatch) 

1 .4.7.5.  *  Input/receive  ACARS  data 
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I.  NORMAL  MISSION  PROFILE 

2.  *  TAKEOFF  OPERATIONS 

2.1.  *  Perform  Takeoff 

2.1.1.  *  Monitor/assess  environmental  factors 

2. 1.1.1.  *  Monitor/assess  weather 

2. 1.1. 2.  *  Maintain  position/terrain/obstacle  awareness 

2. 1.1. 3.  *  Monitor/assess  runway  condition/status 

2.1.2.  *  Reconfigure/Check  Systems 

2. 1.2.1.  *  Perform  procedural  steps 

2. 1.2. 2.  *  Comply  with  reference  info/guidance 

2. 1.2. 3.  Assess/enter/set  FMC/MCP  data 

2. 1.2. 4.  *  Complete  before  takeoff  checklist 

2.1.3.  *  Perform  taxi  on  to  runway 

2. 1.3.1.  *  Perform  thrust  management 

2. 1.3. 2.  *  Perform  brake  management 

2.1 .3.3.  *  Perform  A/C  change  of  control  (if  req) 

2.1 .3.4.  *  Monitor  adjust  nose  wheel  steering 

2. 1.3. 5.  *  Align  aircraft  on  runway  centerline 

2.1.4.  *  Perform  takeoff  roll 

2. 1.4.1.  *  Perform  thrust  management 

2. 1.4. 2.  *  Maintain  runway  alignment/directional  control 

2. 1.4. 3.  *  Maintain  traffic  separation/avoidance 

2. 1.4. 4.  #  Comply  with  reference  info/guidance 

2.1.5.  *  Monitor  takeoff  criteria 

2. 1.5.1.  *  Monitor  systems 

2.1 .5.2.  *  Monitor  external  factors 

2. 1.5. 3.  *  Monitor  aircraft  performance 

2. 1.5.4.  #  Monitor  runway  alignment/directional  track 

2. 1.5. 5.  *  Comply  with  reference  info/guidance 

2.1.6.  *  Perform  rotation/liftoff 

2. 1.6.1.  *  Establish  takeoff  attitude 

2. 1.6. 2.  *  Maintain  runway  alignment  directional  control 

2. 1.6. 3.  *  Comply  with  reference  info/guidance 

2.1.7.  Perform  communications 

2. 1.7.1.  Perform  communication  with  ATC 
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2.1 .7.2.  Perform  cockpit  crew  communication 

2.2.  #  Perform  Rejected  Takeoff 

2.2.1.  *  Monitor/assess  environmental  factors 

2. 2. 1.1.  *  Monitor/assess  weather 

2. 2. 1.2.  *  Maintain  position/terrain/obstacle  awareness 

2. 2. 1.3.  *  Monitor/assess  runway  condition/status 

2.2.2.  *  Assess  abort  decision  factors 

2. 2. 2.1.  *  Monitor/assess  decision  speeds 

2. 2. 2. 2.  *  Assess  systems  function/malfunctioning 

2. 2. 2. 3.  *  Assess  aircraft  controllability 

2. 2. 2.4.  *  Comply  with  reference  info/guidance 

2.2.3.  *  Perform  abort  maneuver 

2.2.3. 1.  *  Perform  brake  management 

2. 2. 3. 2.  *  Monitor  aircraft  performance 

2. 2. 3. 3.  *  Maintain  runway  alignment  directional  control 

2. 2. 3. 4.  *  Perform  thrust  management 

2. 2. 3. 5.  *  Comply  with  reference  info/guidance 

2.2.4.  *  Monitor/check  systems 

2. 2.4.1 .  *  Monitor  hydraulic  systems 

2. 2.4.2.  *  Monitor  electrical  systems 

2.2.5.  *  Reconfigure/check  aircraft 

2.2.5. 1.  *  Perform  procedural  steps 

2.2.6.  *  Perform  post  abort  procedures 

2. 2. 6.1.  *  Assess  A/C  condition 

2. 2. 6. 2.  *  Perform  post  landing  procedural  steps 

2.2.7.  *  Monitor/assess  evacuation  criteria 

2. 2. 7.1.  #  Assess  A/C  condition 

2. 2. 7. 2.  *  Assess  ATC  input 

2. 2. 7. 3.  *  Assess  Flight  Attendant  input 

2.2.8.  Perform  communications 

2.2.8. 1.  *  Perform  communication  with  ATC 

2. 2. 8. 2.  *  Perform  cockpit  crew  communication 

2. 2. 8. 3.  *  Perform  communications  with  cabin  crew 

2. 2. 8. 4.  *  Perform  communications  with  company  (ramp) 

2. 2. 8. 5.  *  Perform  communications  with  maintenance  (Sta) 

2. 2. 8. 6.  *  Perform  communications  with  passengers 
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I.  NORMAL  MISSION  PROFILE 

3.  *  CLIMB  OPERATIONS 

3.1.  *  Perform  initial  climb  to  1000' 

3.1.1.  #  Monitor/assess  environmental  factors 

3. 1 . 1 . 1 .  *  Monitor/assess  weather 

3. 1.1. 2.  *  Maintain  position/terrain  obstacle  awareness 

3.1.2.  *  Reconfigure/check  aircraft 

3. 1 .2. 1 .  *  Perform  procedural  steps 

3. 1.2. 2.  Comply  with  reference  info/guidance 

3.1.3.  *  Fly/monitor/modify  lateral  track 

3.1 .3.1 .  *  Assess/comply  with  radar  vectors 

3. 1.3. 2.  *  Perform  published  departure  procedure 

3.1 .3.3.  Assess/enter/set  MCP  data 

3. 1.3.4.  *  Maintain  traffic  separation/avoidance 

3. 1.3. 5.  Comply  with  reference  info/guidance 

3.1.4.  *  Fly /monitor/modify  vertical  profile 

3. 1.4.1.  *  Monitor/control  airspeed 

3.1 .4.2.  Assess/enter/set  MCP  data 

3. 1.4.3.  *  Perform  published  departure  procedure 

3. 1.4. 4.  *  Comply  with  reference  info/guidance 

3.1.5.  #  Perform  communications 

3. 1.5.1.  #  Perform  communications  with  ATC 

3.1 .5.2.  *  Perform  cockpit  crew  communication 

3. 1.5. 3.  Comply  with  reference  info/guidance 

3.2.  *  Perform  climb  to  3000' 

3.2.1.  *  Monitor/assess  environmental  factors 

3. 2. 1.1.  *  Monitor/assess  weather 

3. 2. 1.2.  *  Maintain  position/terrain/obstacle  awareness 

3.2.2.  *  Reconfigure/check  aircraft 

3.2.2. 1.  *  Perform  procedural  steps 

3. 2. 2. 2.  Comply  with  reference  info/guidance 

3.2.3.  *  Reconfigure/check  systems 

3.2.3. 1.  *  Perform  procedural  flows 

3. 2. 3. 2.  Comply  with  reference  info/guidance 

3.2.4.  *  Fly/monitor/modify  lateral  track 
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3.3. 


3.2. 

3.2 

3.2 

3.2 

3.2. 

3.2. 

3.2.5. 
3.2. 
3.2. 
3.2. 
3.2. 
3.2. 
3.2. 

3.2.6. 
3.2. 
3.2. 


4.1. 

4.2. 

4.3. 

4.4. 

4.5. 

4.6. 


*  Perform  published  departure  procedure 

*  Perform  course  intercept  tracking 

*  Assess/comply  with  radar  vectors 
Assess/enter/set  MCP  data 

*  Maintain  traffic  separation/avoidance 

*  Comply  with  reference  info/guidance 

*  Fly/monitor/modify  vertical  profile 

5.1.  *  Perform  intermediate  level  offs 

*  Perform  published  departure  procedure 

*  Monitor/modify  climb  rate 

*  Monitor/control  airspeed 
Assess/enter/set  FMC/MCP  data 

*  Comply  with  reference  info/guidance 

*  Perform  communications 

6.1.  *  Perform  communications  with  ATC 
Perform  cockpit  crew  communications 


5.2. 

5.3. 

5.4. 

5.5. 

5.6. 


6.2. 


3. 2. 6. 3.  Comply  with  reference  info/guidance 
*  Perform  secondary  climb 

3.3.1.  *  Monitor/assess  environmental  factors 

3. 3. 1.1.  *  Monitor/assess  weather 

3. 3. 1.2.  *  Maintain  position/terrain/obstacle  awareness 

3.3.2.  *  Reconfigure/check  systems 

3. 3. 2.1.  *  Perform  procedural  steps 

3. 3. 2. 2.  Comply  with  reference  info/guidance 

3. 3. 2. 3.  *  Perform  after  takeoff  checklist 

3.3.3.  *  Fly/monitor/modify  lateral  track 

3. 3. 3.1.  *  Perform  published  departure  procedure 

3. 3. 3. 2.  *  Perform  course  intercept  tracking 

3. 3. 3. 3.  *  Assess/comply  with  radar  vectors 

3. 3. 3.4.  Assess/enter/set  FMC/MCP  data 

3. 3. 3. 5.  *  Maintain  traffic  separation/avoidance 

3. 3. 3. 6.  *  Comply  with  reference  info/guidance 

3.3.4.  *  Fly /monitor/modify  vertical  profile 

3. 3.4.1.  *  Perform  intermediate  level  offs 

3. 3.4. 2.  *  Perform  published  departure  procedure 

3. 3. 4. 3.  *  Monitor/modify  climb  rate 
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3. 3. 4. 4.  #  Monitor/control  airspeed 

3. 3. 4. 5.  #  Assess/set  thrust 

3. 3. 4. 6.  Assess/enter/set  FMC/MCP  data 

3. 3. 4. 7.  *  Comply  with  reference  info/guidance 

3.3.5.  *  Perform  communications 

3. 3. 5.1.  *  Perform  communications  with  ATC 

3. 3. 5. 2.  *  Perform  cockpit  crew  communications 

3. 3. 5. 3.  Comply  with  reference  info/guidance 

3. 3. 5.4.  *  Perform  communications  with  cabin  crew 

3. 3. 5. 5.  *  Perform  communications  with  passengers 

3. 3. 5. 6.  *  Input/receive  ACARS  data 
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I.  NORMAL  MISSION  PROFILE 

4.  *  CRUISE  OPERATIONS 

4.1.  *  Perform  enroute  cruise 

4.1.1.  *  Monitor/assess  environmental  factors 

4. 1 . 1 . 1 .  *  Monitor/assess  weather 

4. 1.1.2.  *  Maintain  position/terrain  awareness 

4. 1.1. 3.  *  Comply  with  reference  info/guidance 

4.1.2.  *  Reconfigure/check/monitor  systems 

4. 1.2.1.  *  Perform  periodic/continuing  procedural  steps 

4. 1.2. 2.  *  Comply  with  reference  info/guidance 

4.1.3.  ,*  Fly/monitor/modify  lateral  track 

4.1 .3.1 .  *  Perform  course  intercept  tracking 

4. 1.3. 2.  *  Assess/comply  with  radar  vectors 

4. 1.3. 3.  *  Maintain  traffic  separation/avoidance 

4. 1.3. 4.  Assess/enter/set  FMC/MCP  data 

4. 1.3. 5.  Comply  with  reference  info/guidance 

4.1.4.  *  Fly/monitor/modify  vertical  profile 

4. 1 .4. 1 .  *  Perform  cruise  climbs 

4. 1.4. 2.  *  Preform  cruise  descents 

4. 1.4. 3.  *  Monitor/control  airspeed 

4. 1.4. 4.  Assess/enter/set  FMC/MCP  data 

4. 1.4. 5.  Comply  with  reference  info/guidance 

4.1.5.  *  Anj'lyze/optimize  cruise  parameters 

4. 1.5.1.  Assess  FMC  optimal  cruise  altitudes 

4. 1.5. 2.  *  Assess/set  thrust 

4. 1.5. 3.  *  Assess  alternate  routings 

4. 1.5.4.  *  Monitor/assess  flight  progress 

4. 1.5. 5.  *  Assess  alternate  altitudes 

4. 1.5. 6.  *  Comply  with  reference  info/guidance 

4.1.6.  *  Perform  routine  duties 

4. 1.6.1.  *  Record  pertinent  aircraft  operational  data 

4. 1.6. 2.  *  Comply  with  reference  info/guidance 

4.1.7.  *  Perform  communications 

4. 1.7.1.  *  Perform  communications  with  ATC 

4.1 .7.2.  *  Perform  cockpit  crew  communications 
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4.1. 7. 3.  Comply  with  reference  info/guidance 

4.1. 7.4.  #  Perform  communications  with  cabin  crew 

4. 1.7. 5.  #  Perform  communications  with  company  (dispatch) 

4.1. 7. 6.  *  Perform  communications  with  maintenance  (SAM) 

4. 1.7. 7.  *  Perform  communications  with  passengers 

4. 1.7. 8.  #  Input/receive  ACARS  data 
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I.  NORMAL  MISSION  PROFILE 

5.  *  DESCENT/HOLDING 

5.1.  *  Perform  initial  descent  from  cruise  altitude  to  FL  180 

5.1.1.  *  Monitor/assess  environmental  factors 

5 . 1 . 1 . 1 .  *  Monitor/assess  weather 

5. 1.1. 2.  *  Maintain  position/terrain  awareness 

5.1.2.  *  Reconfigure/check  systems 

5.1 .2.1 .  *  Perform  pre-descent  procedural  steps 

5.1 .2.2.  *  Perform  descent  procedural  steps 

5. 1.2. 3.  *  Comply  with  reference  info/guidance 

5.1.3.  #  Reconfigure/check  aircraft 

5.1 .3.1 .  #  Perform  procedural  steps 

5.1.4.  *  Fly  /monitor/modify  lateral  track 

5.1 .4.1 .  *  Perform  published  arrival/profile  descent  procedure 

5.1 .4.2.  *  Perform  course  intercept  tracking 

5.1 .4.3.  *  Assess/comply  with  radar  vectors 

5. 1.4.4.  Assess/enter/set  FMC/MCP  data 

5. 1.4. 5.  *  Maintain  traffic  separation/avoidance 

5. 1.4. 6.  Comply  with  reference  info/guidance 

5.1.5.  *  Fly/Monitor/modify  vertical  profile 

5. 1.5.1.  *  Perform  cruise  descents 

5. 1.5. 2.  *  Preform  intermediate  leveloffs 

5. 1.5. 3.  *  Perform  published  arrival/profile  descent  procedure 

5. 1.5. 4.  *  Monitor/modify  descent  rate/path 

5. 1.5. 5.  *  Monitor/control  airspeed 

5. 1.5. 6.  Assess/enter/set  FMC/MCP  data 

5. 1.5. 7.  Comply  with  reference  info/guidance 

5.1.6.  *  Perform  communications 

5. 1.6.1.  #  Perform  communications  with  ATC 

5.1 .6.2.  *  Perform  cockpit  crew  communications 

5. 1.6. 3.  *  Comply  with  reference  info/guidance 

5. 1.6. 4.  *  Perform  communications  with  cabin  crew 

5. 1.6. 5.  #  Perform  communications  with  company  (dispatch/ramp) 

5. 1.6. 6.  *  Perform  communications  with  passengers 

5. 1.6. 7.  *  Input/receive  ACARS  data 
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5.2.  #  Perform  approach  descent  from  FL180  to  approach 

5.2.1.  *  Monitor/assess  environmental  factors 

5. 2. 1.1.  *  Monitor/assess  weather 

5. 2. 1.2.  *  Maintain  position/terrain/obstacle  awareness 

5.2.2.  #  Reconfigure/check  systems 

5.2.2. 1.  *  Perform  procedural  steps 

5. 2. 2. 2.  *  Comply  with  reference  info/guidance 

5. 2. 2. 3.  *  Perform  approach/descent  checklist 

5.2.3.  *  Reconfigure/check  aircraft 

5. 2. 3.1.  *  Perform  procedural  steps 

5.2.4.  #  Fly/monitor/modify  lateral  track 

5.2.4. 1 .  *  Perform  published  arrival/profile  descent  procedure  (prior  to  IAF) 

5. 2.4.2.  *  Perform  course  intercept  tracking 

5. 2.4. 3.  *  Assess/comply  with  radar  vectors 

5. 2.4. 4.  Assess/enter/set  FMC/MCP  data 

5. 2.4. 5.  #  Maintain  traffic  separation/avoidance 

5. 2.4. 6.  *  Comply  with  reference  info/guidance 

5.2.5.  *  Fly/Monitor/modify  vertical  profile 

5. 2. 5.1.  *  Perform  approach  descents 

5. 2. 5. 2.  *  Preform  intermediate  leveloffs 

5. 2. 5. 3.  *  Perform  published  arrival/profile  descent  procedure  (prior  to  IAF) 

5. 2. 5. 4.  *  Monitor/modify  descent  rate/path 

5. 2. 5. 5.  *  Monitor/control  airspeed 

5. 2. 5. 6.  Assess/enter/set  FMC/MCP  data 

5. 2. 5. 7.  Comply  with  reference  info/guidance 

5.2.6.  *  Perform  communications 

5. 2. 6.1.  #  Perform  communications  with  ATC 

5. 2. 6. 2.  *  Perform  cockpit  crew  communications 

5. 2. 6.3.  Comply  with  reference  info/guidance 

5. 2. 6.4.  #  Perform  communications  with  cabin  crew 

5. 2. 6. 5.  #  Perform  communications  with  company  (ramp) 

5. 2. 6. 6.  *  Perform  communications  with  passengers 

5. 2. 6. 7.  *  Input/receive  ACARS  data 

5. 2. 6. 8.  *  Comply  with  reference  info/guidance 

5.3.  *  Perform  holding 

5.3.1.  *  Monitor/assess  environmental  factors 
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5. 3. 1.1.  *  Monitor/assess  weather 

5. 3. 1.2.  4  Maintain  position/terrain/obstacle  awareness 

5.3.2.  *  Assess  divert  decision  factors 

5.3.3.  4  Fly/Monitor/modify  lateral  track 

5.3.4.  4  Fly/Monitor/modify  vertical  profile 

5.3.5.  4  Perform  communications 
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I.  NORMAL  MISSION  PROFILE 


6.  *  APPROACH  OPERATIONS 

Note:  The  following  designations  are  used  to  distinguish  which  items  in  the  task  listing  are 
specific  to  a  particular  type  of  instrument  approach.  This  is  done  so  that  all  approaches  can 
be  shown  in  a  single  listing  instead  of  making  a  separate  listing  for  each  individual  approach. 

I  =  Category  I ILS/ILS-DME 

II  =  Category  II  ILS/ILS-DME 

III  =  Category  IIIA/illB  ILS/ILS-DME 
LO  =  LOC/LOC-DME 

BC  =  LOC  Back  Course 
LD  =  LDA/LDA-DME 
VO  =  VOR/VOR-DME 
N  =  NDB/NDB-DME 
S  =  SDF 
A  =  ASR 

V  =  Visual  Approach 
C  =  Contact  Approach 

If  no  designation  is  shown  for  an  action  statement,  then  that  item  is  applicable  to  all  approach 
types. 


6.1.  *  Perform  initial  approach  (Outside  FAF  and/or  prior  to  final  course  intercept) 

6.1.1.  *  Monitor/assess  environmental  factors 

6. 1 . 1 . 1 .  *  Monitor/assess  weather 

6. 1.1. 2.  *  Maintain  position/terrain/obstacle  awareness 

6. 1.1. 3.  *  Comply  with  reference  info/guidance 

6.1.2.  *  Assess  approach  continuation  decision  factors 

6. 1.2.1.  *  Monitor /assess  weather  criteria 

6. 1.2. 2.  *  Monitor/assess  operational  factors 

6. 1.2.3.  *  Monitor /assess  aircraft  status 

6.1 .2.4.  •  Comply  with  reference  info/guidance 

6.1.3.  *  Reconfigure/check  systems 

6. 1.3.1.  *  Perform  procedural  steps 

6.1 .3.2.  Comply  with  reference  info/guidance 

6.1.4.  *  Reconfigure/check  aircraft 

6. 1.4.1.  *  Perform  procedural  steps 
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6. 1.4. 2.  •  Perform  final  descent  checklist 

6. 1.4. 3.  Comply  with  reference  info/guidance 

6.1.5.  *  Fly/monitor/modify  lateral  track 

6. 1.5.1.  *  Determine  position  relative  to  lAF/FAF/procedural  course 

6. 1.5. 2.  *  Perform  published  approach  procedure 

6. 1.5. 3.  *  Monitor/Perform  course  intercept  tracking 

6.1 .5.4.  •  Assess/comply  with  radar  vectors 

6. 1.5. 5.  Assess/enter/set  FMC/MCP  data 

6. 1 .5.6.  Assess  AFDS  requirements 

6. 1.5. 7.  *  Maintain  traffic  separation/avoidance 

6. 1.5.8.  *  Comply  with  reference  info/guidance 

6.1.6.  *  Fly/Monitor/modify  vertical  profile 

6. 1.6.1.  *  Perform  published  approach  procedure 

6. 1.6. 2.  *  Monitor/control  descent  rate/path 

6. 1.6. 3.  *  Monitor/control  airspeed 

6. 1 .6.4.  Assess  AFDS  requirements 

6. 1.6. 5.  Comply  with  reference  info/guidance 

6. 1.6. 6.  Assess/enter/set  FMC/MCP  data 

6.1.7.  *  Perform  communications 

6.1 .7.1 .  *  Perform  communications  with  ATC 

6.1 .7.2.  *  Perform  cockpit  crew  communications 

6. 1.7. 3.  Comply  with  reference  info/guidance 

6. 1.7. 4.  *  Perform  communications  with  cabin  crew 

6.1 .7.5.  *  Perform  communications  with  passengers 

6. 1.7. 6.  Comply  with  reference  info/guidance 

6.2.  *  Perform  final  approach  (after  final  course  intercept  and/or  inside  FAF) 

6.2.1.  *  Monitor/assess  environmental  factors 

6. 2. 1.1.  *  Monitor/assess  weather 

6. 2. 1.2.  *  Maintain  position/terrain/obstacle  awareness 

6.2.2.  *  Assess  continuation/missed  approach  decision  factors  (on  final  approach) 

6.2.2. 1 .  *  Assess  weather  criteria 

6.2. 2. 2.  *  Monitor/assess  operational  factors 

6. 2. 2. 3.  *  Monitor/assess  aircraft  status 

6. 2. 2.4.  *  Comply  with  reference  info/guidance 

6.2.3.  *  Reconfigure/check  systems 

6. 2. 3.1.  *  Perform  procedural  steps 
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6. 2. 3. 2.  Comply  with  reference  info/guidance 

6.2.4.  *  Reconfigure/check  aircraft 

6. 2. 4.1.  *  Perform  procedural  steps 

6. 2. 4. 2.  Comply  with  reference  info/guidance 

6.2.5.  *  Fly/Monitor/modify  lateral  track 

6.2.5. 1.  *  Determine  position  relative  to  procedural  course/MAP 

6. 2. 5. 2.  Perform  published  approach  procedure 

6. 2. 5. 3.  *  *  Monitor/Perform  course  tracking 

6. 2. 5. 4.  *  Assess/comply  with  radar  vectors 

6. 2. 5. 5.  Assess/enter/set  FMC/MCP  data 

6.2. 5.6.  Assess  AFDS  requirements 

6. 2. 5. 7.  *  Maintain  traffic  separation/avoidance 

6. 2. 5. 8.  Comply  with  reference  info/guidance 

6.2.6.  *  Fly/Monitor/modify  vertical  profile 

6.2.6. 1.  *  Perform  published  approach  procedure 

6. 2. 6. 2.  *  Monitor/control  descent  rate/path 

6. 2. 6. 3.  *  Monitor/control  airspeed 

6. 2. 6.4.  Assess  AFDS  requirements 

6. 2. 6. 5.  Assess/enter/set  FMC/MCP  data 

6. 2.6. 6.  Comply  with  reference  info/guidance 

6.2.7.  *  Perform  transition  to  landing/monitor  stabilization/landing  cues 

6. 2. 7.1.  *  Perform  transfer  of  control  (if  req)  (II,  III) 

6. 2. 7. 2.  *  Monitor/assess  weather  criteria 

6. 2. 7. 3.  *  Monitor/assess  operational  factors 

6. 2. 7. 4.  *  Comply  with  reference  info/guidance 

6.2.8.  *  Perform  communications 

6.2.8. 1.  *  Perform  communications  with  ATC 

6. 2. 8. 2.  *  Perform  cockpit  crew  communications 

6. 2. 8. 3.  *  Comply  with  reference  info/guidance 
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*  LANDING  OPERATIONS 

7.1.  *  Landing 

7.1.1.  *  Monitor/assess  environmental  factors 

7. 1.1.1.  *  Monitor /assess  weather 

7. 1.1. 2.  *  Maintain  position/terrain/obstacle  awareness 

7. 1.1. 3.  *  Monitor/assess  runway  condition/status 

7.1.2.  *  Reconfigure/check  systems 

7. 1.2.1.  *  Perform  procedural  steps 

7. 1.2. 2.  Comply  with  reference  info/guidance 

7. 1.2. 3.  *  Monitor  hydraulic  systems 

7.1.3.  *  Reconfigure/check  aircraft 

7. 1.3.1.  *  Perform  procedural  steps 

7. 1.3. 2.  Comply  with  reference  info/guidance 

7.1.4.  *  Perform  landing  maneuver 

7. 1.4.1.  *  Assess/adjust  touchdown  point 

7. 1.4. 2.  *  Perform  flare/touchdown  maneuver 

7. 1.4. 3.  *  Maintain  runway  centerline  alignment/track 

7. 1.4.4.  *  Perform  thrust  management 

7. 1.4. 5.  Comply  with  reference  info/guidance 

7.1.5.  *  Perform  landing  roll  out  maneuver 

7. 1.5.1.  *  Perform  post  touchdown  flight  control  inputs 

7. 1.5. 2.  *  Perform  thrust  management 

7. 1.5. 3.  *  Perform  brake  management 

7. 1.5. 4.  *  Monitor  aircraft  performance 

7. 1.5. 5.  *  Maintain  runway  alignment/directional  control 

7. 1.5. 6.  *  Perform  transfer  of  control  (if  app) 

7. 1.5. 7.  *  Comply  with  reference  info/guidance 

7.1.6.  *  Perform  runway  exit  maneuver 

7. 1.6.1.  *  Perform  thrust  management 

7. 1.6. 2.  *  Perform  brake  management 

7. 1.6. 3.  *  Monitor/adjust  nosewheel  steering 

7.1.7.  *  Perform  communications 

7. 1.7.1.  *  Perform  communications  with  ATC 

7. 1.7. 2.  *  Perform  cockpit  crew  communications 
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7.2.  *  Perform  rejected  landing 

7.2.1.  *  Monitor/assess  environmental  factors 

7. 2. 1.1.  •  Monitor/assess  weather 

7. 2. 1.2.  *  Maintain  position/terrain/obstacle  awareness 

7.2.2.  *  Assess  go  around  decision  factors 

7. 2. 2.1.  *  Monitor/assess  weather  criteria 

7. 2. 2. 2.  *  Monitor/assess  operational  factors 

7. 2. 2. 3.  *  Monitor/assess  aircraft  status 

7.2.3.  *  Reconfigure/check  systems 

7. 2. 3.1.  *  Perform  procedural  steps 

7. 2. 3. 2.  Comply  with  reference  info/guidance 

7. 2. 3. 3.  *  Perform  after  takeoff  checklist 

7.2.4.  *  Reconfigure/check  aircraft 

7.2.4. 1.  *  Perform  procedural  steps 

7.2.5.  *  Fly/Monitor/modify  lateral  track 

7. 2.5.1.  *  Perform  course  intercept  tracking 

7. 2. 5. 2.  #  Assess/comply  with  radar  vectors 

7. 2. 5. 3.  *  Perform  published  missed  approach  procedure 

7. 2. 5. 4.  Assess/enter/set  MCP  data 

7. 2. 5. 5.  *  Maintain  traffic  separation/avoidance 

7. 2. 5. 6.  *  Comply  with  reference  info/guidance 

7.2.6.  *  Fly/Monitor/modify  vertical  profile 

7.2.6. 1 .  Establish/modify  target  pitch  attitudes 

7. 2. 6. 2.  *  Monitor/control  airspeed 

7. 2. 6. 3.  *  Monitor/modify  climb  rate 

7. 2. 6. 4.  Assess/enter/set  MCP  data 

7. 2. 6. 5.  *  Perform  published  missed  approach  procedure 

7. 2. 6. 6.  Monitor/respond  to  autopilot  engagement  mode  effects 

7. 2. 6. 7.  Comply  with  reference  info/guidance 

7.2.7.  *  Perform  communications 

7.2.7. 1.  •  Perform  communications  with  ATC 

7. 2. 7. 2.  *  Perform  cockpit  crew  communications 

7. 2. 7. 3.  *  Perform  communications  with  cabin  crew 

7. 2. 7. 4.  *  Perform  communications  with  company  (dispatch,  if  req) 

7. 2. 7. 5.  *  Perform  communications  with  passengers 

7. 2. 7. 6.  *  Input/receive  ACARS  data 
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I.  NORMAL  MISSION  PROFILE 

8.  *  POST  FLIGHT  GROUND  OPERATIONS 

8.1.  *  Perform  taxi 

8.1.1.  *  Monitor/assess  environmental  factors 

8. 1.1.1.  *  Monitor/assess  weather 

8. 1.1. 2.  *  Maintain  position/terrain/obstacle  awareness 

8. 1.1. 3.  *  Monitor/assess  taxiway/ramp  condition/suitability 

8.1.2.  *  Reconfigure/check  systems 

8. 1.2.1.  *  Perform  procedural  flows 

8. 1.2. 2.  Comply  with  reference  info/guidance 

8.1.3.  *  Reconfigure/check  aircraft 

8. 1.3.1.  *  Perform  procedural  flows 

8. 1.3. 2.  *  Comply  with  reference  info/guidance 

8.1.4.  *  Maneuver  aircraft 

8.1 .4.1 .  *  Perform  thrust  management 

8. 1.4. 2.  *  Perform  brake  management 

8. 1.4. 3.  *  Monitor/control  directional  steering 

8. 1.4. 4.  *  Comply  with  reference  info/guidance 

8.1.5.  *  Perform  airport  navigation 

8.1 .5.1 .  vMonitor  airport  ground  traffic 

8.1 .5.2.  *  Identify  taxi  route 

8 . 1 . 5 . 3.  Review/assess  airport  chart 

8. 1.5. 4.  *  Assess/comply  with  ATC  instructions 

8.1.6.  *  Perform  Communications 

8. 1.6.1.  *  Perform  communications  with  ATC 

8.1 .6.2.  *  Perform  cockpit  crew  communications 

8.1 .6.3.  *  Perform  communications  with  cabin  crew 

8. 1.6. 4.  *  Perform  communications  with  company  (ramp) 

8. 1.6. 5.  *  Perform  communications  with  passengers 

8. 1.6. 6.  *  Input/receive  ACARS  data 

8.2.  *  Perform  parking 

8.2.1.  *  Monitor/assess  environmental  factors 

8. 2. 1.1.  *  Monitor/assess  weather 

8. 2. 1.2.  *  Maintain  position/obstacle  awareness 

8. 2. 1.3.  *  Monitor/assess  parking  area  condition/suitability 
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8. 2.1.4. 

8.2.2.  *  Reconfigure/check  systems 

8. 2. 2.1.  *  Perform  procedural  steps 

8. 2. 2. 2.  Comply  with  reference  info/guidance 

8.2.3.  *  Maneuver  aircraft 

8. 2. 3.1.  *  Perform  thrust  management 

8. 2. 3. 2.  *  Perform  brake  management 

8. 2. 3. 3.  *  Monitor/control  directional  steering 

8. 2. 3. 4.  *  Align/stop  aircraft 

8. 2. 3. 5.  *  Comply  with  reference  info/guidance 

8.2.4.  *  *  *  *  Perform  communications 

8. 2. 4.1.  Perform  cockpit  crew  communication 

8.3.  #  Perform  shutdown 

8.3.1.  *  *  Reconfigure/check  systems 

8. 3. 1.1.  Perform  normal  shutdown  procedural  flows 

8. 3. 1.2.  *  Perform  "terminating  flight"  procedural  flows 

8. 3. 1.3.  Comply  with  reference  info/guidance 

8. 3. 1.4.  *  Perform  parking  checklist 

8.3.2.  *  Perform  communications 

8.3.2. 1 .  *  Perform  cockpit  crew  communications 

8.4.  *  Perform  post  shutdown 

8.4.1.  *  Perform  routine  duties 

8.4.1 .1 .  *  Monitor/assess  aircraft  systems  status 

8. 4. 1.2.  *  Record  pertinent  aircraft  operational  data 

8.4.2.  *  Perform  communications 

8. 4. 2.1.  *  Perform  cockpit  crew  communication 

8. 4. 2. 2.  *  Perform  communications  with  cabin 

8. 4. 2. 3.  *  Perform  communications  with  company  (dispatch/ramp) 

8. 4. 2. 4.  *  Perform  communications  with  maintenance  (station/SAM) 

8. 4. 2. 5.  *  Perform  communications  with  passengers 

8. 4. 2. 6.  *  Perform  communications  with  passenger  service  agent 

8. 4.2. 7.  Input/receive  ACARS  data 
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TASK  ANALYSIS  STRUCTURING 

The  following  are  definitions  of  different  levels  of  the  task  analysis  structure  used  by  United 
Airlines  in  their  AQP  application.  As  much  as  possible.  United  Airlines  attempts  to  conform 
to  this  guideline  when  assigning  individual  action  statements  from  a  job  inventory  to 
different  levels  of  the  task  analysis. 

PHASE: 

Phase  of  flight  (ground  operations,  takeoff,  etc.},  organized  in  sequence.  (Note:  no  action 
verb  required.) 

PROCEDURE: 

General  subset  of  phase,  typically  sequential  or  chronological,  i.e.: 

1)  Climb  to  1000'. 

2)  Climb  to  3000'. 

3)  Secondary  climb,  etc. 

(Note:  action  verb  required.) 

TASK: 

Sets  of  "action  groups"  which  are  non-sequential,  not  necessarily  related,  all  of  which  must 
be  accomplished  to  some  degree  to  successfully  perform  the  parent  procedure,  i.e.: 

1)  Perform  communications. 

2)  Reconfigure/check  aircraft. 

3)  Fly/monitor/modify  lateral  track,  etc. 

(Note:  action  verb  required.) 

SUBTASK: 

Specific  subset  of  non-sequential,  potentially  related  "action  groups"  any  or  all  of  which 
may  need  to  be  accomplished  to  successfully  complete  the  parent  task.  Some  subtasks 
may  be  optional  or  may  be  performed  to  the  exclusion  of  another  subtask  listed  under  the 
same  parent  task,  i.e.: 

1)  Perform  course  intercept/tracking. 

2)  Assess/comply  with  radar  vectors. 

3)  Maintain  traffic  separation/avoidance. 

(Note:  action  verb  required.) 

ELEMENT: 

Specific  set  of  discrete  or  grouped  actions  or  non-action  inventory  list  items  which  may  be 
a  breakdown  of  a  subtask  single  action  statement.  An  element  usually  specifies  a  discrete 
activity  or  sets  of  related  or  non-related  discrete  activities  to  support  the  subtask  action 
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group.  Typical  examples  are  a  statement  of  a  single  switch  movement  or  a  statement  of 
positioning  of  a  switch  group  with  individual  switch  names  inventoried  in  the  sub-element 
group.  Other  element  level  activities  may  include  cognitive  functions  such  as  monitor, 
consider  or  determine  certain  items  which  may  affect  the  accomplishment  of  the  subtask, 
i.e.: 

1)  Set  crossfeed  switch. 

2)  Set  fuel  pump  switches: 

•  main  pumps 

•  center  tank  pumps 

3)  Determine  crew  duties. 

4)  Assess  conflicting  traffic  inputs: 

•  visual 

•  ATC 

•  other  Aircraft 

•  TCAS. 

(Note:  Action  verb  may  be  required.) 

SUPPLEMENT: 

sets  of  related  non-sequential  inventoried  items  which  are  supporting  lists  to  action 
statements  of  the  element  level.  May  include  action  statement,  but  only  on  the  very  lowest 
performance  level,  i.e.: 

1 )  Determine  crew  duties  (element) 

•  aircraft  control  (sub-element) 

•  checklist  accomplishment 

•  communication  assignments 
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APPENDIX  E: 

Synopsis  of  Jan  Demuth's  Comments 
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SYNOPSIS  OF  JAN  DEMUTH'S  COMMENTS 
21  November  1991  -  Integrated  CRM  Workshop 


These  remarks  were  made  in  the  context  of  organizational  culture  and  support  for  CRM. 
The  main  points  follow. 

(1 )  CRM  skills  and  technical  skills  can  be  and  should  integrated--at  least  in  the  teaching 
mode. 

(2)  At  this  point  in  time,  and  with  the  revised  CRM  advisory,  we  have  some  common 
language  and  a  start-point  exists. 

(3)  The  revised  CRM  advisory  is  a  bench  mark  for  where  we  (FAA)  are  now-and,  the 
revision  also  reflects  that  industry  was  long  past  the  "old"  CRM  advisory  being 
useful. 

(4)  The  interchange,  communication  and  cooperation  between  FAA,  industry  and 
academe  which  has  occurred  due  to  the  ATA  subcommittee  and  working  group- 
and,  in  the  drafting  of  the  revisions  to  the  CRM  advisory-is  of  a  high  level.  Our 
hope  is  that  this  cooperation  and  involvement  will  continue. 

(5)  While  no  airline  must  use  anything  in  the  revised  advisory,  we  hope  the  common 
language  and  cooperative  efforts  that  led  to  this  revision  will  facilitate  the  airlines 
in  using  the  advisory  as  they  see  fit. 

Mr.  Demuth  then  re-addressed  a  hot  issue  at  the  workshop:  CRM  Evaluation.  He  again  said 
that  CRM  is  not  being  used  by  the  FAA  for  qualification  and  certification  issues.  When/if 
the  evaluation  issues  can  be  resolved,  the  rule  will  be  made-a  rule  based  on  what  has  been 
learned,  what  can  be  done  and  validated  as  to  CRM  evaluation.  He  also  re-made  the 
distinction  between  mandatory  CRM  and  mandating  its  evaluation.  He  did  indicate  the 
industry  must  look  at  Integrated  CRM  Evaluation  as  it  will  come  about. 
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APPENDIX  F: 

Results  of  Survey  Forms 
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Participant  Data  Sheet  (Phase  I  -  APPENDIX  C) 

Unless  otherwise  noted,  all  figures  are  averages  of  participants'  responses. 
LENGTH  OF  TIME  WITH  CURRENT  ORGANIZATION:  13.44  years 


PILOT  (sum):  Y  _8_  N  _2_ 

TYPE  OF  LICENSE  (sum):  8  ATPs 


CURRENTLY  LICENSED  (sum):  Y  _8_  N 


AIRCRAFT  QUALIFIED  IN  (list  &  [number  of  pilots]):  Average  =  3.38  types  per  pilot. 


B-52  [1 


B-19QO  ri 


DC-3  n 


B-737J2 


B747-400  [1 


B757/767  [3 


Sabreliner  65  fl 


T-39I1 


HAVE  EXPERIENCE  IN:  (count) 


(average) 


Lis  Y  7 


Length  of  Time  9.; 


Y  JL  Length  of  Time  3.44  years 


Y_5_ 


Length  of  Time  5.11  years 


Y  8 


Length  of  Time  6.5  years 


Y  6 


Length  of  Time  5.22  years 
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PRE-WORKSHOP  SURVEY 


(Averages) 

1 .  My  skills  in  analyzing  and  evaluating  put  me  about  here,  relative  to  the  others. 
Very  highly  skilled  No  skill  at  all 

1  2  3  4  5  6  7  3.3 


2.  I  think  my  ideas  will  be,  in  essence,  in  agreement  with  the  rest  of  the 
participants. 

Yes,  absolutely  No,  not  at  all 

1  2  3  4  5  6  7 


3.0 


3.  I  know  most  of  the  participants  very  well. 

Yes,  pretty  much  No,  none  at  all 

1  2  3  4  5  6  7 


4.7 


4. 


I  have  some  definite  ideas  about  what  skills  are  necessary  for  aviation  mission 
success. 

Yes,  lots  No,  none 

1  2  3  4  5  6  7  2.1 


5.  I  have  extensive  experience  in  aviation  operations  analysis. 

Yes  No 

1  2  3  4  5  6  7 


3.6 


6.  I  have  extensive  experience  in  training  and  evaluating  aviators. 

Yes  No 

1  2  3  4  5  6  7 


2.0 


7.  I  am  anticipating  that  the  workshop  is  going  to  be  achieving  our  objectives. 
Yes,  I  think  it  No,  I  think  it  may 

will  be  be  a  waste  of  time 


1  2  3  4  5  6  7 


2.1 
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POST  WORKSHOP  SURVEY 


1 .  I  feel  very  satisfied 
with  the  results. 


2.  I  learned  from  the 
feedback. 


3.  In  general,  I  agreed 
with  the  ideas  in 
the  feedback. 


(Averages) 

1  2  3  4  5  6  7 

2.7 

1  2  3  4  5  6  7 

1.5 

1  2  3  4  5  6  7 

1.8 


I'm  not  satisfied  with 
the  results,  at  all. 


I  didn't  learn  a  thing 
from  the  feedback. 


I  disagreed  with 
everything  in  the 
feedback. 


4.  I  felt  that  I  could  1  2  3  4  5  6  7  I  couldn't  really  say 

express  my  ideas  what  I  wanted  to  say. 

well.  1.5 


5.  I  felt  as  if  I  really 
wanted  to  talk  or 
interact. 


1  2  3  4  5  6  7 

2.0 


I  didn't  feel  the  need 
to  talk  or  interact  at 
all. 


6.  I  have  a  feeling  the 
participants  didn't 
understand  or  think 
about  my  reasons. 


1  2  3  4  5  6  7 

6.0 


I  think  the  participants 
understood  my 
reasons  pretty  well. 


7.  I  think  the 

workshop  went  too 
quickly;  we  needed 
more  time. 


1  2  3  4  5  6  7 

3.8 


I  think  the  workshop 
went  too  slowly;  we 
could've 

accomplished  the 
objectives  with  much 
less  time. 
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GROUP  PROCESSES  SURVEY 


A.  How  clear  are  the  group  goals? 

1 .  No  apparent  goals _ 

2.  Goal  confusion,  uncertainty,  or  conflict 

3.  Average  goal  clarity _ 

4.  Goals  mostly  clear _ 

5>.  Goals  very  clear _ 


(Averages) 

3.5 


B.  How  much  trust  and  openness  in  the  group?  4.3 

1 .  Distrust,  a  closed  group _ 

2.  Little  trust,  defensiveness _ 

3.  Average  trust  and  openness _ 

4.  Considerable  trust  and  openness _ 

5.  Remarkable  trust  and  openness _ 


C.  How  sensitive  and  perceptive  are  group  members?  4.2 

1 .  No  awareness  or  listening  in  the  group _ 

2.  Most  members  self-absorbed _ 

3.  Average  sensitivity  and  listening _ 

4.  Better  than  usual  listening _ 

5.  Outstanding  sensitivity  to  others _ 


D.  How  much  attention  was  paid  to  process  (the  way  the  group  was  working)?  3.3 

1 .  No  attention  to  process _ 

2.  Little  attention  to  process _ 

3.  Some  concern  with  group  process _ 

4.  A  fair  balance  between  content  and  process _ 

5.  Very  concerned  with  process _ 


E.  How  were  group  leadership  needs  met?  4.5 

1 .  Not  met,  drifting _ 

2.  Leadership  concentrated  in  one  person _ 

3.  Some  leadership  sharing _ 

4.  Leadership  functions  distributed _ 

5.  Leadership  needs  met  creatively  and  flexibly _ 
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F.  How  were  group  decisions  made?  4.3 

1 .  No  decisions  could  be  reached _ 

2.  Made  by  a  few _ 

3.  Majority  vote _ 

4.  Attempts  at  integrating  minority  vote _ 

5.  Full  participation  and  tested  consensus _ 


G.  How  well  were  group  resources  used?  4.0 

1 .  One  or  two  contributed,  but  most  were  silent _ 

2.  Several  tried  to  contribute,  but  were  discouraged _ 

3.  Average  use  of  group  resources _ 

4.  Group  resources  well  used  and  encouraged _ 

5.  Group  resources  fully  and  effectively  used _ 


H.  How  much  loyalty  and  sense  of  belonging  to  the  group?  4.2 

1 .  Members  had  no  group  loyalty  or  sense  of  belonging _ 

2.  Members  not  close,  but  some  friendly  relations _ 

3.  About  average  sense  of  belonging _ 

4.  Some  warm  sense  of  belonging _ 

5.  Strong  sense  of  belonging  among  members _ 


